{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# Learning MLP using Fashion Mnist dataset with ChatGPT.ipynb"
      ],
      "metadata": {
        "id": "6dQSoNw5TQ4w"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Sure, let's start with the first step: Data Pre-processing.\n",
        "\n",
        "### Data Pre-processing:\n",
        "Data pre-processing is a crucial step in machine learning, where we prepare our data for training by applying various transformations and normalization techniques. In this step, we'll normalize the pixel values, flatten the images into vectors, and encode categorical labels.\n",
        "\n",
        "#### Normalize Pixel Values:\n",
        "Normalizing pixel values ensures that each pixel's value is scaled to a range between 0 and 1. This is important for the neural network to converge faster during training and to prevent any particular feature from dominating the learning process.\n",
        "\n",
        "#### Flatten Images into Vectors:\n",
        "MLP models cannot directly work with multi-dimensional data like images. Hence, we need to flatten the images into 1D vectors, where each pixel value becomes a feature.\n",
        "\n",
        "#### Encode Categorical Labels:\n",
        "Since our labels are categorical (e.g., different types of clothing items), we need to encode them into numerical values so that our model can understand and learn from them.\n",
        "\n",
        "Now, let's implement these steps in code:\n",
        "\n",
        "```python\n",
        "import numpy as np\n",
        "from tensorflow.keras.datasets import fashion_mnist\n",
        "from sklearn.preprocessing import StandardScaler, LabelEncoder\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "# Step 1: Load Fashion MNIST dataset\n",
        "(x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()\n",
        "\n",
        "# Step 2: Normalize pixel values\n",
        "def normalize_images(train_data, test_data):\n",
        "    # Convert to float32 to ensure division results in float\n",
        "    train_data = train_data.astype('float32')\n",
        "    test_data = test_data.astype('float32')\n",
        "    # Normalize pixel values to the range 0-1\n",
        "    train_data /= 255.0\n",
        "    test_data /= 255.0\n",
        "    return train_data, test_data\n",
        "\n",
        "# Step 3: Flatten images into vectors\n",
        "def flatten_images(train_data, test_data):\n",
        "    # Reshape images to vectors\n",
        "    train_data = train_data.reshape(train_data.shape[0], -1)\n",
        "    test_data = test_data.reshape(test_data.shape[0], -1)\n",
        "    return train_data, test_data\n",
        "\n",
        "# Step 4: Encode categorical labels\n",
        "def encode_labels(train_labels, test_labels):\n",
        "    # Initialize LabelEncoder\n",
        "    label_encoder = LabelEncoder()\n",
        "    # Fit LabelEncoder on training labels and transform both training and testing labels\n",
        "    train_labels = label_encoder.fit_transform(train_labels)\n",
        "    test_labels = label_encoder.transform(test_labels)\n",
        "    return train_labels, test_labels\n",
        "\n",
        "# Apply data pre-processing steps\n",
        "x_train, x_test = normalize_images(x_train, x_test)\n",
        "x_train, x_test = flatten_images(x_train, x_test)\n",
        "y_train, y_test = encode_labels(y_train, y_test)\n",
        "\n",
        "# Print the shapes of pre-processed data\n",
        "print(\"x_train shape:\", x_train.shape)\n",
        "print(\"y_train shape:\", y_train.shape)\n",
        "print(\"x_test shape:\", x_test.shape)\n",
        "print(\"y_test shape:\", y_test.shape)\n",
        "```\n",
        "\n",
        "Explanation:\n",
        "- We import necessary libraries such as NumPy for numerical operations, TensorFlow for loading the Fashion MNIST dataset, and scikit-learn for data preprocessing.\n",
        "- We load the Fashion MNIST dataset using the `fashion_mnist.load_data()` method and split it into training and testing sets.\n",
        "- Three functions `normalize_images`, `flatten_images`, and `encode_labels` are defined to perform each data preprocessing step.\n",
        "- Inside each function:\n",
        "  - `normalize_images` divides pixel values by 255 to scale them between 0 and 1.\n",
        "  - `flatten_images` reshapes the images into 1D vectors.\n",
        "  - `encode_labels` converts categorical labels into numerical values using `LabelEncoder`.\n",
        "- Finally, we apply these preprocessing steps to the training and testing data and print their shapes to ensure everything is processed correctly."
      ],
      "metadata": {
        "id": "vh8Q8f0_TFzY"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import numpy as np\n",
        "from tensorflow.keras.datasets import fashion_mnist\n",
        "from sklearn.preprocessing import StandardScaler, LabelEncoder\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "# Step 1: Load Fashion MNIST dataset\n",
        "(x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()\n",
        "\n",
        "# Step 2: Normalize pixel values\n",
        "def normalize_images(train_data, test_data):\n",
        "    # Convert to float32 to ensure division results in float\n",
        "    train_data = train_data.astype('float32')\n",
        "    test_data = test_data.astype('float32')\n",
        "    # Normalize pixel values to the range 0-1\n",
        "    train_data /= 255.0\n",
        "    test_data /= 255.0\n",
        "    return train_data, test_data\n",
        "\n",
        "# Step 3: Flatten images into vectors\n",
        "def flatten_images(train_data, test_data):\n",
        "    # Reshape images to vectors\n",
        "    train_data = train_data.reshape(train_data.shape[0], -1)\n",
        "    test_data = test_data.reshape(test_data.shape[0], -1)\n",
        "    return train_data, test_data\n",
        "\n",
        "# Step 4: Encode categorical labels\n",
        "def encode_labels(train_labels, test_labels):\n",
        "    # Initialize LabelEncoder\n",
        "    label_encoder = LabelEncoder()\n",
        "    # Fit LabelEncoder on training labels and transform both training and testing labels\n",
        "    train_labels = label_encoder.fit_transform(train_labels)\n",
        "    test_labels = label_encoder.transform(test_labels)\n",
        "    return train_labels, test_labels\n",
        "\n",
        "# Apply data pre-processing steps\n",
        "x_train, x_test = normalize_images(x_train, x_test)\n",
        "x_train, x_test = flatten_images(x_train, x_test)\n",
        "y_train, y_test = encode_labels(y_train, y_test)\n",
        "\n",
        "# Print the shapes of pre-processed data\n",
        "print(\"x_train shape:\", x_train.shape)\n",
        "print(\"y_train shape:\", y_train.shape)\n",
        "print(\"x_test shape:\", x_test.shape)\n",
        "print(\"y_test shape:\", y_test.shape)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1GO3LCrzU2Xt",
        "outputId": "053f81dd-8a1e-4065-8174-315930b5e5be"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-labels-idx1-ubyte.gz\n",
            "29515/29515 [==============================] - 0s 0us/step\n",
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-images-idx3-ubyte.gz\n",
            "26421880/26421880 [==============================] - 1s 0us/step\n",
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-labels-idx1-ubyte.gz\n",
            "5148/5148 [==============================] - 0s 0us/step\n",
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-images-idx3-ubyte.gz\n",
            "4422102/4422102 [==============================] - 0s 0us/step\n",
            "x_train shape: (60000, 784)\n",
            "y_train shape: (60000,)\n",
            "x_test shape: (10000, 784)\n",
            "y_test shape: (10000,)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Great! Let's move on to the next step:\n",
        "\n",
        "### Data Splitting:\n",
        "In this step, we'll partition the dataset into training, validation, and testing sets. The training set will be used to train the model, the validation set will be used to tune hyperparameters and prevent overfitting, and the testing set will be used to evaluate the model's performance on unseen data.\n",
        "\n",
        "#### Training Set:\n",
        "The training set is used to train the model's parameters (weights and biases) using optimization algorithms like gradient descent.\n",
        "\n",
        "#### Validation Set:\n",
        "The validation set is used to evaluate the model's performance during training and adjust hyperparameters such as learning rate, batch size, and number of epochs to optimize the model's performance.\n",
        "\n",
        "#### Testing Set:\n",
        "The testing set is used to evaluate the final performance of the trained model on unseen data and assess its generalization capability.\n",
        "\n",
        "Now, let's split the dataset:\n",
        "\n",
        "```python\n",
        "# Step 5: Split dataset into training, validation, and testing sets\n",
        "def split_dataset(train_data, train_labels, test_data, test_labels, validation_size=0.1, test_size=0.1, random_state=42):\n",
        "    # Split training set into training and validation sets\n",
        "    x_train, x_val, y_train, y_val = train_test_split(train_data, train_labels, test_size=validation_size, random_state=random_state)\n",
        "    # Split testing set into testing and validation sets\n",
        "    x_test, x_val, y_test, y_val = train_test_split(test_data, test_labels, test_size=test_size, random_state=random_state)\n",
        "    return x_train, y_train, x_val, y_val, x_test, y_test\n",
        "\n",
        "# Apply data splitting\n",
        "x_train, y_train, x_val, y_val, x_test, y_test = split_dataset(x_train, y_train, x_test, y_test)\n",
        "\n",
        "# Print the shapes of split datasets\n",
        "print(\"Training set:\")\n",
        "print(\"x_train shape:\", x_train.shape)\n",
        "print(\"y_train shape:\", y_train.shape)\n",
        "print(\"\\nValidation set:\")\n",
        "print(\"x_val shape:\", x_val.shape)\n",
        "print(\"y_val shape:\", y_val.shape)\n",
        "print(\"\\nTesting set:\")\n",
        "print(\"x_test shape:\", x_test.shape)\n",
        "print(\"y_test shape:\", y_test.shape)\n",
        "```\n",
        "\n",
        "Explanation:\n",
        "- We define a function `split_dataset` to split the dataset into training, validation, and testing sets using the `train_test_split` function from scikit-learn.\n",
        "- The function takes training data, training labels, testing data, testing labels, and optional parameters such as validation size, test size, and random state.\n",
        "- We split the training set into training and validation sets using the specified validation size.\n",
        "- Then, we split the testing set into testing and validation sets using the specified test size.\n",
        "- Finally, we print the shapes of the split datasets to verify the split."
      ],
      "metadata": {
        "id": "7EWDaODwezqi"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Step 5: Split dataset into training, validation, and testing sets\n",
        "def split_dataset(train_data, train_labels, test_data, test_labels, validation_size=0.1, test_size=0.1, random_state=42):\n",
        "    # Split training set into training and validation sets\n",
        "    x_train, x_val, y_train, y_val = train_test_split(train_data, train_labels, test_size=validation_size, random_state=random_state)\n",
        "    # Split testing set into testing and validation sets\n",
        "    x_test, x_val, y_test, y_val = train_test_split(test_data, test_labels, test_size=test_size, random_state=random_state)\n",
        "    return x_train, y_train, x_val, y_val, x_test, y_test\n",
        "\n",
        "# Apply data splitting\n",
        "x_train, y_train, x_val, y_val, x_test, y_test = split_dataset(x_train, y_train, x_test, y_test)\n",
        "\n",
        "# Print the shapes of split datasets\n",
        "print(\"Training set:\")\n",
        "print(\"x_train shape:\", x_train.shape)\n",
        "print(\"y_train shape:\", y_train.shape)\n",
        "print(\"\\nValidation set:\")\n",
        "print(\"x_val shape:\", x_val.shape)\n",
        "print(\"y_val shape:\", y_val.shape)\n",
        "print(\"\\nTesting set:\")\n",
        "print(\"x_test shape:\", x_test.shape)\n",
        "print(\"y_test shape:\", y_test.shape)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ggXQqiZee0co",
        "outputId": "e5738130-0c4d-4311-f7a7-737e565845ea"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Training set:\n",
            "x_train shape: (54000, 784)\n",
            "y_train shape: (54000,)\n",
            "\n",
            "Validation set:\n",
            "x_val shape: (1000, 784)\n",
            "y_val shape: (1000,)\n",
            "\n",
            "Testing set:\n",
            "x_test shape: (9000, 784)\n",
            "y_test shape: (9000,)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Sure, let's move on to the next step:\n",
        "\n",
        "### Model Selection:\n",
        "In this step, we'll choose a Multi-Layer Perceptron (MLP) as the classification model. An MLP consists of one or more layers of neurons, each fully connected to the next layer. For simplicity, we'll start with a single-layer MLP as a baseline model.\n",
        "\n",
        "#### Single-Layer MLP:\n",
        "A single-layer MLP consists of an input layer, a hidden layer, and an output layer. However, since it has only one hidden layer, it's often referred to as a single-layer perceptron. Each neuron in the hidden layer receives input from all neurons in the input layer and produces an output, which is then passed through an activation function to introduce non-linearity. Finally, the output layer produces the final prediction.\n",
        "\n",
        "Now, let's define the baseline single-layer MLP model:\n",
        "\n",
        "```python\n",
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import Dense\n",
        "\n",
        "# Step 6: Define a single-layer MLP model\n",
        "def create_single_layer_mlp(input_shape, num_classes):\n",
        "    model = Sequential()\n",
        "    # Add a single hidden layer with ReLU activation\n",
        "    model.add(Dense(128, activation='relu', input_shape=input_shape))\n",
        "    # Add output layer with softmax activation for multi-class classification\n",
        "    model.add(Dense(num_classes, activation='softmax'))\n",
        "    return model\n",
        "\n",
        "# Define input shape and number of classes\n",
        "input_shape = x_train.shape[1:]\n",
        "num_classes = len(np.unique(y_train))\n",
        "\n",
        "# Create the single-layer MLP model\n",
        "model = create_single_layer_mlp(input_shape, num_classes)\n",
        "\n",
        "# Print model summary\n",
        "model.summary()\n",
        "```\n",
        "\n",
        "Explanation:\n",
        "- We import necessary modules from TensorFlow.keras to define the MLP model.\n",
        "- We define a function `create_single_layer_mlp` to create a single-layer MLP model.\n",
        "- Inside the function:\n",
        "  - We initialize a sequential model.\n",
        "  - We add a single hidden layer with 128 neurons and ReLU activation function.\n",
        "  - We add an output layer with softmax activation function for multi-class classification.\n",
        "- We specify the input shape based on the shape of the training data and the number of output classes.\n",
        "- We create the single-layer MLP model using the defined function.\n",
        "- Finally, we print the summary of the model architecture."
      ],
      "metadata": {
        "id": "jzFvCiXIhGfC"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import Dense\n",
        "\n",
        "# Step 6: Define a single-layer MLP model\n",
        "def create_single_layer_mlp(input_shape, num_classes):\n",
        "    model = Sequential()\n",
        "    # Add a single hidden layer with ReLU activation\n",
        "    model.add(Dense(128, activation='relu', input_shape=input_shape))\n",
        "    # Add output layer with softmax activation for multi-class classification\n",
        "    model.add(Dense(num_classes, activation='softmax'))\n",
        "    return model\n",
        "\n",
        "# Define input shape and number of classes\n",
        "input_shape = x_train.shape[1:]\n",
        "num_classes = len(np.unique(y_train))\n",
        "\n",
        "# Create the single-layer MLP model\n",
        "model = create_single_layer_mlp(input_shape, num_classes)\n",
        "\n",
        "# Print model summary\n",
        "model.summary()\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-fmaJI6WhHJ4",
        "outputId": "a29a826b-28c0-47db-e3f7-4b1ea4df21f3"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"sequential\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " dense (Dense)               (None, 128)               100480    \n",
            "                                                                 \n",
            " dense_1 (Dense)             (None, 10)                1290      \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 101770 (397.54 KB)\n",
            "Trainable params: 101770 (397.54 KB)\n",
            "Non-trainable params: 0 (0.00 Byte)\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Sure, let's move on to the next step:\n",
        "\n",
        "### Model Training:\n",
        "In this step, we'll train the MLP model on the training data. Training involves feeding the training data to the model, adjusting its parameters (weights and biases) using optimization algorithms like stochastic gradient descent, and minimizing a loss function to make predictions as close to the actual labels as possible.\n",
        "\n",
        "#### Loss Function:\n",
        "The loss function measures the difference between the predicted labels and the actual labels. For multi-class classification problems like Fashion MNIST, we typically use categorical cross-entropy as the loss function.\n",
        "\n",
        "#### Optimization Algorithm:\n",
        "The optimization algorithm is responsible for updating the model's parameters during training to minimize the loss function. We'll use Adam optimizer, which is an extension to stochastic gradient descent and is widely used in deep learning.\n",
        "\n",
        "#### Training Parameters:\n",
        "We'll specify training parameters such as batch size and number of epochs. Batch size determines the number of samples processed before updating the model's parameters, while the number of epochs specifies how many times the entire training dataset is passed forward and backward through the model.\n",
        "\n",
        "Now, let's train the model:\n",
        "\n",
        "```python\n",
        "from tensorflow.keras.utils import to_categorical\n",
        "\n",
        "# Step 7: Model Training\n",
        "def train_model(model, x_train, y_train, x_val, y_val, batch_size=32, epochs=10):\n",
        "    # Convert class vectors to binary class matrices (one-hot encoding)\n",
        "    y_train = to_categorical(y_train)\n",
        "    y_val = to_categorical(y_val)\n",
        "    # Compile the model with categorical cross-entropy loss and Adam optimizer\n",
        "    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])\n",
        "    # Train the model on the training data\n",
        "    history = model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, validation_data=(x_val, y_val))\n",
        "    return history\n",
        "\n",
        "# Train the model\n",
        "history = train_model(model, x_train, y_train, x_val, y_val, batch_size=32, epochs=10)\n",
        "```\n",
        "\n",
        "Explanation:\n",
        "- We import necessary modules from TensorFlow.keras to compile and train the model.\n",
        "- We define a function `train_model` to train the model on the training data.\n",
        "- Inside the function:\n",
        "  - We convert the class labels to binary class matrices using one-hot encoding.\n",
        "  - We compile the model with categorical cross-entropy loss function and Adam optimizer.\n",
        "  - We train the model on the training data with specified batch size and number of epochs.\n",
        "- Finally, we train the model using the defined function and store the training history for visualization and analysis."
      ],
      "metadata": {
        "id": "SRQD13Yfijnt"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from tensorflow.keras.utils import to_categorical\n",
        "\n",
        "# Step 7: Model Training\n",
        "def train_model(model, x_train, y_train, x_val, y_val, batch_size=32, epochs=10):\n",
        "    # Convert class vectors to binary class matrices (one-hot encoding)\n",
        "    y_train = to_categorical(y_train)\n",
        "    y_val = to_categorical(y_val)\n",
        "    # Compile the model with categorical cross-entropy loss and Adam optimizer\n",
        "    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])\n",
        "    # Train the model on the training data\n",
        "    history = model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, validation_data=(x_val, y_val))\n",
        "    return history\n",
        "\n",
        "# Train the model\n",
        "history = train_model(model, x_train, y_train, x_val, y_val, batch_size=32, epochs=10)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "J2VG8swRik2p",
        "outputId": "e1def8a0-5d3f-4a83-a4e8-09f78087b7a1"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "1688/1688 [==============================] - 6s 3ms/step - loss: 0.5038 - accuracy: 0.8231 - val_loss: 0.4535 - val_accuracy: 0.8210\n",
            "Epoch 2/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.3781 - accuracy: 0.8646 - val_loss: 0.3870 - val_accuracy: 0.8550\n",
            "Epoch 3/10\n",
            "1688/1688 [==============================] - 6s 3ms/step - loss: 0.3392 - accuracy: 0.8765 - val_loss: 0.3674 - val_accuracy: 0.8690\n",
            "Epoch 4/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.3127 - accuracy: 0.8854 - val_loss: 0.3454 - val_accuracy: 0.8780\n",
            "Epoch 5/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2960 - accuracy: 0.8910 - val_loss: 0.3372 - val_accuracy: 0.8680\n",
            "Epoch 6/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2803 - accuracy: 0.8958 - val_loss: 0.3497 - val_accuracy: 0.8720\n",
            "Epoch 7/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2666 - accuracy: 0.8997 - val_loss: 0.3657 - val_accuracy: 0.8650\n",
            "Epoch 8/10\n",
            "1688/1688 [==============================] - 6s 3ms/step - loss: 0.2554 - accuracy: 0.9039 - val_loss: 0.3464 - val_accuracy: 0.8700\n",
            "Epoch 9/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2459 - accuracy: 0.9079 - val_loss: 0.3492 - val_accuracy: 0.8700\n",
            "Epoch 10/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2382 - accuracy: 0.9106 - val_loss: 0.3363 - val_accuracy: 0.8800\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Great! Let's move on to the next step:\n",
        "\n",
        "### Model Evaluation:\n",
        "In this step, we'll use various metrics to evaluate the performance of the trained MLP model on the testing data. Commonly used metrics for classification tasks include accuracy, precision, recall, and the confusion matrix.\n",
        "\n",
        "#### Accuracy:\n",
        "Accuracy measures the proportion of correctly classified instances out of the total instances.\n",
        "\n",
        "#### Precision:\n",
        "Precision measures the proportion of true positive predictions out of all positive predictions. It indicates the model's ability to avoid false positives.\n",
        "\n",
        "#### Recall:\n",
        "Recall measures the proportion of true positive predictions out of all actual positive instances. It indicates the model's ability to identify all positive instances.\n",
        "\n",
        "#### Confusion Matrix:\n",
        "A confusion matrix provides a summary of the model's performance by showing the number of true positive, true negative, false positive, and false negative predictions.\n",
        "\n",
        "Now, let's evaluate the model:\n",
        "\n",
        "```python\n",
        "import matplotlib.pyplot as plt\n",
        "from sklearn.metrics import accuracy_score, precision_score, recall_score, confusion_matrix\n",
        "\n",
        "# Step 8: Model Evaluation\n",
        "def evaluate_model(model, x_test, y_test):\n",
        "    # Predict labels for testing data\n",
        "    y_pred = model.predict(x_test)\n",
        "    # Convert predicted labels to class labels\n",
        "    y_pred_classes = np.argmax(y_pred, axis=1)\n",
        "    # Convert true labels to class labels\n",
        "    y_true_classes = np.argmax(y_test, axis=1)\n",
        "    # Calculate accuracy\n",
        "    accuracy = accuracy_score(y_true_classes, y_pred_classes)\n",
        "    # Calculate precision\n",
        "    precision = precision_score(y_true_classes, y_pred_classes, average='weighted')\n",
        "    # Calculate recall\n",
        "    recall = recall_score(y_true_classes, y_pred_classes, average='weighted')\n",
        "    # Calculate confusion matrix\n",
        "    cm = confusion_matrix(y_true_classes, y_pred_classes)\n",
        "    return accuracy, precision, recall, cm\n",
        "\n",
        "# Evaluate the model\n",
        "accuracy, precision, recall, cm = evaluate_model(model, x_test, y_test)\n",
        "\n",
        "# Print evaluation metrics\n",
        "print(\"Accuracy:\", accuracy)\n",
        "print(\"Precision:\", precision)\n",
        "print(\"Recall:\", recall)\n",
        "print(\"Confusion Matrix:\\n\", cm)\n",
        "\n",
        "# Visualize confusion matrix\n",
        "plt.imshow(cm, cmap=plt.cm.Blues)\n",
        "plt.title('Confusion Matrix')\n",
        "plt.colorbar()\n",
        "plt.xlabel('Predicted Labels')\n",
        "plt.ylabel('True Labels')\n",
        "plt.show()\n",
        "```\n",
        "\n",
        "Explanation:\n",
        "- We import necessary modules from matplotlib.pyplot and scikit-learn to visualize the confusion matrix and calculate evaluation metrics.\n",
        "- We define a function `evaluate_model` to evaluate the model's performance on the testing data.\n",
        "- Inside the function:\n",
        "  - We make predictions on the testing data using the trained model.\n",
        "  - We convert the predicted labels and true labels to class labels.\n",
        "  - We calculate accuracy, precision, recall, and confusion matrix using scikit-learn metrics functions.\n",
        "- Finally, we evaluate the model using the defined function, print the evaluation metrics, and visualize the confusion matrix."
      ],
      "metadata": {
        "id": "Q_2eI_NakWA5"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import matplotlib.pyplot as plt\n",
        "from sklearn.metrics import accuracy_score, precision_score, recall_score, confusion_matrix\n",
        "\n",
        "# Step 8: Model Evaluation\n",
        "def evaluate_model(model, x_test, y_test):\n",
        "    # Predict labels for testing data\n",
        "    y_pred = model.predict(x_test)\n",
        "    # Convert predicted labels to class labels\n",
        "    y_pred_classes = np.argmax(y_pred, axis=1)\n",
        "    # Convert true labels to class labels\n",
        "    y_true_classes = np.argmax(y_test, axis=1)\n",
        "    # Calculate accuracy\n",
        "    accuracy = accuracy_score(y_true_classes, y_pred_classes)\n",
        "    # Calculate precision\n",
        "    precision = precision_score(y_true_classes, y_pred_classes, average='weighted')\n",
        "    # Calculate recall\n",
        "    recall = recall_score(y_true_classes, y_pred_classes, average='weighted')\n",
        "    # Calculate confusion matrix\n",
        "    cm = confusion_matrix(y_true_classes, y_pred_classes)\n",
        "    return accuracy, precision, recall, cm\n",
        "\n",
        "# Evaluate the model\n",
        "accuracy, precision, recall, cm = evaluate_model(model, x_test, y_test)\n",
        "\n",
        "# Print evaluation metrics\n",
        "print(\"Accuracy:\", accuracy)\n",
        "print(\"Precision:\", precision)\n",
        "print(\"Recall:\", recall)\n",
        "print(\"Confusion Matrix:\\n\", cm)\n",
        "\n",
        "# Visualize confusion matrix\n",
        "plt.imshow(cm, cmap=plt.cm.Blues)\n",
        "plt.title('Confusion Matrix')\n",
        "plt.colorbar()\n",
        "plt.xlabel('Predicted Labels')\n",
        "plt.ylabel('True Labels')\n",
        "plt.show()\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 383
        },
        "id": "QKKdoHijkX4D",
        "outputId": "f547e520-5f7d-4e1a-943a-205ab60f3227"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "282/282 [==============================] - 2s 5ms/step\n"
          ]
        },
        {
          "output_type": "error",
          "ename": "AxisError",
          "evalue": "axis 1 is out of bounds for array of dimension 1",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mAxisError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-5-02193580c831>\u001b[0m in \u001b[0;36m<cell line: 23>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     21\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     22\u001b[0m \u001b[0;31m# Evaluate the model\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 23\u001b[0;31m \u001b[0maccuracy\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mprecision\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrecall\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcm\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mevaluate_model\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx_test\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my_test\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     24\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     25\u001b[0m \u001b[0;31m# Print evaluation metrics\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m<ipython-input-5-02193580c831>\u001b[0m in \u001b[0;36mevaluate_model\u001b[0;34m(model, x_test, y_test)\u001b[0m\n\u001b[1;32m      9\u001b[0m     \u001b[0my_pred_classes\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0my_pred\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     10\u001b[0m     \u001b[0;31m# Convert true labels to class labels\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 11\u001b[0;31m     \u001b[0my_true_classes\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0my_test\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     12\u001b[0m     \u001b[0;31m# Calculate accuracy\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     13\u001b[0m     \u001b[0maccuracy\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0maccuracy_score\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0my_true_classes\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my_pred_classes\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/numpy/core/fromnumeric.py\u001b[0m in \u001b[0;36margmax\u001b[0;34m(a, axis, out, keepdims)\u001b[0m\n\u001b[1;32m   1227\u001b[0m     \"\"\"\n\u001b[1;32m   1228\u001b[0m     \u001b[0mkwds\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m'keepdims'\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mkeepdims\u001b[0m\u001b[0;34m}\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mkeepdims\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_NoValue\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1229\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0m_wrapfunc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'argmax'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0maxis\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mout\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mout\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1230\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1231\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/numpy/core/fromnumeric.py\u001b[0m in \u001b[0;36m_wrapfunc\u001b[0;34m(obj, method, *args, **kwds)\u001b[0m\n\u001b[1;32m     57\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     58\u001b[0m     \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 59\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mbound\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     60\u001b[0m     \u001b[0;32mexcept\u001b[0m \u001b[0mTypeError\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     61\u001b[0m         \u001b[0;31m# A TypeError occurs if the object does have such a method in its\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mAxisError\u001b[0m: axis 1 is out of bounds for array of dimension 1"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "It seems like there's an issue with the shape of the labels. The error indicates that the axis specified for the `argmax` function is out of bounds. This typically occurs when the labels are not in the expected format or shape.\n",
        "\n",
        "Let's double-check the shape of the labels and make sure they are in the correct format for evaluation:\n",
        "\n",
        "```python\n",
        "print(\"Shape of y_test:\", y_test.shape)\n",
        "```\n",
        "\n",
        "This will help us understand the shape of the labels and identify any discrepancies. Once we identify the issue, we can correct it accordingly."
      ],
      "metadata": {
        "id": "acqDcH0gohV0"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "print(\"Shape of y_test:\", y_test.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "e0YtWeLnovd7",
        "outputId": "96b44a21-7e94-4cfe-a6af-7c7a8968b209"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Shape of y_test: (9000,)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Thank you for providing the shape of `y_test`. It seems that the labels are in 1D format, which means they are not one-hot encoded. In the evaluation function, we assumed that the labels are one-hot encoded, which caused the error.\n",
        "\n",
        "To resolve this issue, we need to modify the evaluation function to handle 1D labels instead of one-hot encoded labels. We can achieve this by removing the conversion of labels to one-hot encoding and directly compare the predicted labels with the true labels.\n",
        "\n",
        "Let's update the evaluation function accordingly:\n",
        "\n",
        "```python\n",
        "# Step 8: Model Evaluation\n",
        "def evaluate_model(model, x_test, y_test):\n",
        "    # Predict labels for testing data\n",
        "    y_pred = model.predict(x_test)\n",
        "    # Convert predicted labels to class labels\n",
        "    y_pred_classes = np.argmax(y_pred, axis=1)\n",
        "    # Calculate accuracy\n",
        "    accuracy = accuracy_score(y_test, y_pred_classes)\n",
        "    # Calculate precision\n",
        "    precision = precision_score(y_test, y_pred_classes, average='weighted')\n",
        "    # Calculate recall\n",
        "    recall = recall_score(y_test, y_pred_classes, average='weighted')\n",
        "    # Calculate confusion matrix\n",
        "    cm = confusion_matrix(y_test, y_pred_classes)\n",
        "    return accuracy, precision, recall, cm\n",
        "\n",
        "# Evaluate the model\n",
        "accuracy, precision, recall, cm = evaluate_model(model, x_test, y_test)\n",
        "\n",
        "# Print evaluation metrics\n",
        "print(\"Accuracy:\", accuracy)\n",
        "print(\"Precision:\", precision)\n",
        "print(\"Recall:\", recall)\n",
        "print(\"Confusion Matrix:\\n\", cm)\n",
        "\n",
        "# Visualize confusion matrix\n",
        "plt.imshow(cm, cmap=plt.cm.Blues)\n",
        "plt.title('Confusion Matrix')\n",
        "plt.colorbar()\n",
        "plt.xlabel('Predicted Labels')\n",
        "plt.ylabel('True Labels')\n",
        "plt.show()\n",
        "```\n",
        "\n",
        "Explanation:\n",
        "- We removed the conversion of labels to one-hot encoding and directly used the true labels `y_test` in the evaluation function.\n",
        "- We updated the accuracy, precision, and recall calculation functions to use the 1D labels instead of one-hot encoded labels.\n",
        "- Now, the evaluation function should work correctly with the provided 1D labels, and the evaluation metrics should be calculated accurately."
      ],
      "metadata": {
        "id": "x_IstMtDpiEy"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Step 8: Model Evaluation\n",
        "def evaluate_model(model, x_test, y_test):\n",
        "    # Predict labels for testing data\n",
        "    y_pred = model.predict(x_test)\n",
        "    # Convert predicted labels to class labels\n",
        "    y_pred_classes = np.argmax(y_pred, axis=1)\n",
        "    # Calculate accuracy\n",
        "    accuracy = accuracy_score(y_test, y_pred_classes)\n",
        "    # Calculate precision\n",
        "    precision = precision_score(y_test, y_pred_classes, average='weighted')\n",
        "    # Calculate recall\n",
        "    recall = recall_score(y_test, y_pred_classes, average='weighted')\n",
        "    # Calculate confusion matrix\n",
        "    cm = confusion_matrix(y_test, y_pred_classes)\n",
        "    return accuracy, precision, recall, cm\n",
        "\n",
        "# Evaluate the model\n",
        "accuracy, precision, recall, cm = evaluate_model(model, x_test, y_test)\n",
        "\n",
        "# Print evaluation metrics\n",
        "print(\"Accuracy:\", accuracy)\n",
        "print(\"Precision:\", precision)\n",
        "print(\"Recall:\", recall)\n",
        "print(\"Confusion Matrix:\\n\", cm)\n",
        "\n",
        "# Visualize confusion matrix\n",
        "plt.imshow(cm, cmap=plt.cm.Blues)\n",
        "plt.title('Confusion Matrix')\n",
        "plt.colorbar()\n",
        "plt.xlabel('Predicted Labels')\n",
        "plt.ylabel('True Labels')\n",
        "plt.show()\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 740
        },
        "id": "5NTVnvgjpiVb",
        "outputId": "05055400-9db8-47fa-8299-b04e7cf9d2cb"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "282/282 [==============================] - 1s 3ms/step\n",
            "Accuracy: 0.8795555555555555\n",
            "Precision: 0.8817019102604995\n",
            "Recall: 0.8795555555555555\n",
            "Confusion Matrix:\n",
            " [[683   2  15  25   5   3 175   0   8   0]\n",
            " [  0 881   1  12   3   0   3   0   1   0]\n",
            " [  7   1 707   9 103   1  60   0   1   0]\n",
            " [ 15  15  10 804  28   1  27   0   6   0]\n",
            " [  0   3  72  37 753   0  40   0   1   0]\n",
            " [  0   0   0   0   0 866   0  11   0  17]\n",
            " [ 63   1  79  29  83   0 646   0   6   0]\n",
            " [  0   0   0   0   0  20   0 850   0  25]\n",
            " [  3   0   2   3   5   3   8   3 877   0]\n",
            " [  0   0   0   0   0   5   1  27   0 849]]\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 2 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfsAAAHHCAYAAAC4M/EEAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABKtklEQVR4nO3de1xUdf4/8NcMyIDADEJyWxHxkoqieEtHykuSZOjqypYUKd7LwFTS1F2veKHcVNIQ0p8pmWzllpZoKmJqJipilpfykiR80wFTYUTlInN+f7jMOg7mDDMwl/N6+jiPbT7nc855n3mwvPl8zud8PhJBEAQQERGR3ZJaOgAiIiKqX0z2REREdo7JnoiIyM4x2RMREdk5JnsiIiI7x2RPRERk55jsiYiI7ByTPRERkZ1jsiciIrJzTPZED7lw4QIGDhwIhUIBiUSCbdu2mfX8v/32GyQSCTZu3GjW89qyfv36oV+/fpYOg8huMdmTVfr111/x2muvoWXLlnB2doZcLkdYWBjef/993L17t16vHRsbi1OnTmHJkiXYtGkTunfvXq/Xa0ijR4+GRCKBXC6v9Xu8cOECJBIJJBIJ3nvvPaPPf+XKFSxYsAAnT540Q7REZC6Olg6A6GE7duzAiy++CJlMhlGjRqFjx46orKzEoUOHMGPGDJw5cwZr166tl2vfvXsXOTk5+Oc//4n4+Ph6uUZgYCDu3r2LRo0a1cv5H8fR0RF37tzB9u3b8dJLL+ns27x5M5ydnVFeXl6nc1+5cgULFy5EixYtEBoaavBxe/bsqdP1iMgwTPZkVfLz8xEdHY3AwEDs27cPfn5+2n1xcXG4ePEiduzYUW/Xv3btGgDAw8Oj3q4hkUjg7Oxcb+d/HJlMhrCwMPz73//WS/YZGRmIjIzEF1980SCx3LlzB40bN4aTk1ODXI9IrNiNT1Zl2bJlKCsrw/r163USfY3WrVtjypQp2s/37t3DokWL0KpVK8hkMrRo0QL/+Mc/UFFRoXNcixYtMHjwYBw6dAhPPfUUnJ2d0bJlS3z88cfaOgsWLEBgYCAAYMaMGZBIJGjRogWA+93fNf/9oAULFkAikeiUZWVl4emnn4aHhwfc3NzQtm1b/OMf/9Duf9Qz+3379uGZZ56Bq6srPDw8MHToUPz888+1Xu/ixYsYPXo0PDw8oFAoMGbMGNy5c+fRX+xDXnnlFXzzzTcoKSnRluXm5uLChQt45ZVX9OrfuHED06dPR0hICNzc3CCXyzFo0CD8+OOP2jr79+9Hjx49AABjxozRPg6ouc9+/fqhY8eOyMvLQ58+fdC4cWPt9/LwM/vY2Fg4Ozvr3X9ERASaNGmCK1euGHyvRMRkT1Zm+/btaNmyJXr37m1Q/fHjx2PevHno2rUrVq5cib59+yIpKQnR0dF6dS9evIi///3veO6557B8+XI0adIEo0ePxpkzZwAAw4cPx8qVKwEAL7/8MjZt2oTk5GSj4j9z5gwGDx6MiooKJCYmYvny5fjrX/+K77///k+P27t3LyIiIlBcXIwFCxYgISEBhw8fRlhYGH777Te9+i+99BJu3bqFpKQkvPTSS9i4cSMWLlxocJzDhw+HRCLBl19+qS3LyMhAu3bt0LVrV736ly5dwrZt2zB48GCsWLECM2bMwKlTp9C3b19t4m3fvj0SExMBABMnTsSmTZuwadMm9OnTR3ue69evY9CgQQgNDUVycjL69+9fa3zvv/8+mjZtitjYWFRXVwMAPvzwQ+zZswerV6+Gv7+/wfdKRAAEIitRWloqABCGDh1qUP2TJ08KAITx48frlE+fPl0AIOzbt09bFhgYKAAQDh48qC0rLi4WZDKZ8NZbb2nL8vPzBQDCv/71L51zxsbGCoGBgXoxzJ8/X3jw/0YrV64UAAjXrl17ZNw119iwYYO2LDQ0VPD29hauX7+uLfvxxx8FqVQqjBo1Su96Y8eO1Tnn3/72N8HLy+uR13zwPlxdXQVBEIS///3vwoABAwRBEITq6mrB19dXWLhwYa3fQXl5uVBdXa13HzKZTEhMTNSW5ebm6t1bjb59+woAhLS0tFr39e3bV6ds9+7dAgBh8eLFwqVLlwQ3Nzdh2LBhj71HItLHlj1ZDbVaDQBwd3c3qP7OnTsBAAkJCTrlb731FgDoPdsPDg7GM888o/3ctGlTtG3bFpcuXapzzA+redb/1VdfQaPRGHTM1atXcfLkSYwePRqenp7a8k6dOuG5557T3ueDXn/9dZ3PzzzzDK5fv679Dg3xyiuvYP/+/VCpVNi3bx9UKlWtXfjA/ef8Uun9XxfV1dW4fv269hHFiRMnDL6mTCbDmDFjDKo7cOBAvPbaa0hMTMTw4cPh7OyMDz/80OBrEdH/MNmT1ZDL5QCAW7duGVT/8uXLkEqlaN26tU65r68vPDw8cPnyZZ3y5s2b652jSZMmuHnzZh0j1jdixAiEhYVh/Pjx8PHxQXR0ND7//PM/Tfw1cbZt21ZvX/v27fHHH3/g9u3bOuUP30uTJk0AwKh7eeGFF+Du7o7PPvsMmzdvRo8ePfS+yxoajQYrV65EmzZtIJPJ8MQTT6Bp06b46aefUFpaavA1//KXvxg1GO+9996Dp6cnTp48iVWrVsHb29vgY4nof5jsyWrI5XL4+/vj9OnTRh338AC5R3FwcKi1XBCEOl+j5nlyDRcXFxw8eBB79+7FyJEj8dNPP2HEiBF47rnn9OqawpR7qSGTyTB8+HCkp6dj69atj2zVA8DSpUuRkJCAPn364JNPPsHu3buRlZWFDh06GNyDAdz/fozxww8/oLi4GABw6tQpo44lov9hsierMnjwYPz666/Iycl5bN3AwEBoNBpcuHBBp7yoqAglJSXakfXm0KRJE52R6zUe7j0AAKlUigEDBmDFihU4e/YslixZgn379uHbb7+t9dw1cZ47d05v3y+//IInnngCrq6upt3AI7zyyiv44YcfcOvWrVoHNdb4z3/+g/79+2P9+vWIjo7GwIEDER4ervedGPqHlyFu376NMWPGIDg4GBMnTsSyZcuQm5trtvMTiQmTPVmVt99+G66urhg/fjyKior09v/66694//33AdzvhgagN2J+xYoVAIDIyEizxdWqVSuUlpbip59+0pZdvXoVW7du1al348YNvWNrJpd5+HXAGn5+fggNDUV6erpO8jx9+jT27Nmjvc/60L9/fyxatAgffPABfH19H1nPwcFBr9dgy5Yt+P3333XKav4oqe0PI2PNnDkTBQUFSE9Px4oVK9CiRQvExsY+8nskokfjpDpkVVq1aoWMjAyMGDEC7du315lB7/Dhw9iyZQtGjx4NAOjcuTNiY2Oxdu1alJSUoG/fvjh27BjS09MxbNiwR77WVRfR0dGYOXMm/va3v+HNN9/EnTt3kJqaiieffFJngFpiYiIOHjyIyMhIBAYGori4GGvWrEGzZs3w9NNPP/L8//rXvzBo0CAolUqMGzcOd+/exerVq6FQKLBgwQKz3cfDpFIp5syZ89h6gwcPRmJiIsaMGYPevXvj1KlT2Lx5M1q2bKlTr1WrVvDw8EBaWhrc3d3h6uqKnj17IigoyKi49u3bhzVr1mD+/PnaVwE3bNiAfv36Ye7cuVi2bJlR5yMSPQu/DUBUq/PnzwsTJkwQWrRoITg5OQnu7u5CWFiYsHr1aqG8vFxbr6qqSli4cKEQFBQkNGrUSAgICBBmz56tU0cQ7r96FxkZqXedh1/5etSrd4IgCHv27BE6duwoODk5CW3bthU++eQTvVfvsrOzhaFDhwr+/v6Ck5OT4O/vL7z88svC+fPn9a7x8Otpe/fuFcLCwgQXFxdBLpcLQ4YMEc6ePatTp+Z6D7/at2HDBgGAkJ+f/8jvVBB0X717lEe9evfWW28Jfn5+gouLixAWFibk5OTU+srcV199JQQHBwuOjo4699m3b1+hQ4cOtV7zwfOo1WohMDBQ6Nq1q1BVVaVTb9q0aYJUKhVycnL+9B6ISJdEEIwY0UNEREQ2h8/siYiI7ByTPRERkZ1jsiciIrJzTPZERER2jsmeiIjIzjHZExER2TmbnlRHo9HgypUrcHd3N+s0nURE1DAEQcCtW7fg7++vXVmxPpSXl6OystLk8zg5OcHZ2dkMETUsm072V65cQUBAgKXDICIiExUWFqJZs2b1cu7y8nK4uHsB9+6YfC5fX1/k5+fbXMK36WRfs+65z6h1kDo1tnA0/3Py3cGWDkHPvWrDVyZrKBornM6pkYP19RBZY6/VhauGLUPckNr4uVs6BD0aK/whl0qt6+fpllqN1kEB2t/n9aGyshK4dwey4FjAwfAllvVUV0J1Nh2VlZVM9g2p5peg1KmxVSX7mnXZrQmTvWGY7A3jVmZ9McnlTPaGsLZkX6NBfs4dnSExIdkLEtsd5mbTyZ6IiMhgEgCm/FFhnX8nGYTJnoiIxEEivb+ZcryNst3IiYiIyCBs2RMRkThIJCZ249tuPz6TPRERiQO78YmIiMhesWVPRETiwG58IiIie2diN74Nd4bbbuRERERkELbsiYhIHNiNT0REZOc4Gt+yUlJS0KJFCzg7O6Nnz544duyYpUMiIiKyGxZP9p999hkSEhIwf/58nDhxAp07d0ZERASKi4stHRoREdmTmm58UzYbZfFkv2LFCkyYMAFjxoxBcHAw0tLS0LhxY3z00UeWDo2IiOxJTTe+KZuNsmjklZWVyMvLQ3h4uLZMKpUiPDwcOTk5FoyMiIjsjohb9hYdoPfHH3+guroaPj4+OuU+Pj745Zdf9OpXVFSgoqJC+1mtVtd7jERERLbOpvokkpKSoFAotFtAQIClQyIiIlvBbnzLeOKJJ+Dg4ICioiKd8qKiIvj6+urVnz17NkpLS7VbYWFhQ4VKRES2TiIxMdnbbje+RZO9k5MTunXrhuzsbG2ZRqNBdnY2lEqlXn2ZTAa5XK6zERER0Z+z+KQ6CQkJiI2NRffu3fHUU08hOTkZt2/fxpgxYywdGhER2ROp5P5myvE2yuLJfsSIEbh27RrmzZsHlUqF0NBQ7Nq1S2/QHhERkUlEPIOexZM9AMTHxyM+Pt7SYRAREdklq0j2RERE9Y4L4RAREdk5EXfj227kREREZBC27ImISBzYjU9ERGTn2I1PRERk5xp4IZzq6mrMnTsXQUFBcHFxQatWrbBo0SIIgqCtIwgC5s2bBz8/P7i4uCA8PBwXLlzQOc+NGzcQExMDuVwODw8PjBs3DmVlZUbFwmRPRERUD959912kpqbigw8+wM8//4x3330Xy5Ytw+rVq7V1li1bhlWrViEtLQ1Hjx6Fq6srIiIiUF5erq0TExODM2fOICsrC5mZmTh48CAmTpxoVCzsxiciInFo4G78w4cPY+jQoYiMjAQAtGjRAv/+979x7NgxAPdb9cnJyZgzZw6GDh0KAPj444/h4+ODbdu2ITo6Gj///DN27dqF3NxcdO/eHQCwevVqvPDCC3jvvffg7+9vUCxs2RMRkTiYqRtfrVbrbA8uvf6g3r17Izs7G+fPnwcA/Pjjjzh06BAGDRoEAMjPz4dKpUJ4eLj2GIVCgZ49eyInJwcAkJOTAw8PD22iB4Dw8HBIpVIcPXrU4Ftny56IiMgIDy+vPn/+fCxYsECv3qxZs6BWq9GuXTs4ODiguroaS5YsQUxMDABApVIBgN708D4+Ptp9KpUK3t7eOvsdHR3h6emprWMIJnsiIhIJU9ekv39sYWGhzqqrMpms1tqff/45Nm/ejIyMDHTo0AEnT57E1KlT4e/vj9jYWBPiMB6TPRERiYOZ3rM3dIn1GTNmYNasWYiOjgYAhISE4PLly0hKSkJsbCx8fX0BAEVFRfDz89MeV1RUhNDQUACAr68viouLdc5779493LhxQ3u8Iewi2Z98d7BVrW3fpIf1LepzM/cDS4egp1ojPL5SA5PY8KQZDamtv7ulQ7AJUhteEpVMd+fOHUiluj0JDg4O0Gg0AICgoCD4+voiOztbm9zVajWOHj2KSZMmAQCUSiVKSkqQl5eHbt26AQD27dsHjUaDnj17GhyLXSR7IiKix5JITByNb9wfb0OGDMGSJUvQvHlzdOjQAT/88ANWrFiBsWPH/vd0EkydOhWLFy9GmzZtEBQUhLlz58Lf3x/Dhg0DALRv3x7PP/88JkyYgLS0NFRVVSE+Ph7R0dEGj8QHmOyJiEgsGvjVu9WrV2Pu3Ll44403UFxcDH9/f7z22muYN2+ets7bb7+N27dvY+LEiSgpKcHTTz+NXbt2wdnZWVtn8+bNiI+Px4ABAyCVShEVFYVVq1YZF7rw4FQ+NkatVkOhUKDoeim78R+D3fiGcWC3K1GDUqvV8PFSoLS0/n6P1+QKWcR7kDRyqfN5hKq7qNg9vV5jrS9s2RMRkThwIRwiIiI7J+KFcJjsiYhIHETcsrfdP1OIiIjIIGzZExGROLAbn4iIyM6xG5+IiIjsFVv2REQkChKJxLQpsW24Zc9kT0REoiDmZM9ufCIiIjvHlj0REYmD5L+bKcfbKCZ7IiISBXbjExERkd1iy56IiERBzC17JnsiIhIFJnsiIiI7J+Zkz2f2REREdo4teyIiEge+ekdERGTf2I1PREREdosteyIiEoX7K9ya0rI3XywNjcmeiIhEQQITu/FtONuzG5+IiMjOsWVPRESiIOYBekz2REQkDiJ+9Y7d+ERERHaOLXsiIhIHE7vxBXbjExERWTdTn9mbNpLfspjsiYhIFMSc7PnMnoiIyM6xZU9EROLA0fhERET2raYb35TNGC1atKj1HHFxcQCA8vJyxMXFwcvLC25uboiKikJRUZHOOQoKChAZGYnGjRvD29sbM2bMwL1794y+dyZ7IiKiepCbm4urV69qt6ysLADAiy++CACYNm0atm/fji1btuDAgQO4cuUKhg8frj2+uroakZGRqKysxOHDh5Geno6NGzdi3rx5RsciEQRBMM9tNTy1Wg2FQoGr10ogl8stHY6WVGp9fT2Br2+xdAh68tf83dIh6Ll2q8LSIejxUThbOgQ96rtVlg5Bj9ylkaVDoDpQq9Xw8VKgtLS03n6P1+SKpqPSIXVqXOfzaCrv4NrHsXWOderUqcjMzMSFCxegVqvRtGlTZGRk4O9/v/+78JdffkH79u2Rk5ODXr164ZtvvsHgwYNx5coV+Pj4AADS0tIwc+ZMXLt2DU5OTgZfmy17IiISBXN146vVap2touLxjYTKykp88sknGDt2LCQSCfLy8lBVVYXw8HBtnXbt2qF58+bIyckBAOTk5CAkJESb6AEgIiICarUaZ86cMeremeyJiIiMEBAQAIVCod2SkpIee8y2bdtQUlKC0aNHAwBUKhWcnJzg4eGhU8/HxwcqlUpb58FEX7O/Zp8xOBqfiIhEwVzv2RcWFup048tkssceu379egwaNAj+/v51vr4pmOyJiEgczPTqnVwuN+qZ/eXLl7F37158+eWX2jJfX19UVlaipKREp3VfVFQEX19fbZ1jx47pnKtmtH5NHUOxG5+IiKgebdiwAd7e3oiMjNSWdevWDY0aNUJ2dra27Ny5cygoKIBSqQQAKJVKnDp1CsXFxdo6WVlZkMvlCA4ONioGtuyJiEgULDFdrkajwYYNGxAbGwtHx/+lXIVCgXHjxiEhIQGenp6Qy+WYPHkylEolevXqBQAYOHAggoODMXLkSCxbtgwqlQpz5sxBXFycQY8OHsRkT0REomCJZL93714UFBRg7NixevtWrlwJqVSKqKgoVFRUICIiAmvWrNHud3BwQGZmJiZNmgSlUglXV1fExsYiMTHR6DiY7ImISBQskewHDhyIR01n4+zsjJSUFKSkpDzy+MDAQOzcudPo6z6Mz+yJiIjsHFv2REQkDiJeCIfJnoiIRIHr2RMREZHdYsueiIhEgS17C0lKSkKPHj3g7u4Ob29vDBs2DOfOnbNkSEREZKckMHEhHBt+aG/RZH/gwAHExcXhyJEjyMrKQlVVFQYOHIjbt29bMiwiIiK7YtFu/F27dul83rhxI7y9vZGXl4c+ffpYKCoiIrJHYu7Gt6pn9qWlpQAAT09PC0dCRER2h6/eWZ5Go8HUqVMRFhaGjh071lqnoqICFRUV2s9qtbqhwiMiIrJZVvPqXVxcHE6fPo1PP/30kXWSkpKgUCi0W0BAQANGSEREtsykwXkmPgKwNKtI9vHx8cjMzMS3336LZs2aPbLe7NmzUVpaqt0KCwsbMEoiIrJlYk72Fu3GFwQBkydPxtatW7F//34EBQX9aX2ZTGb0sn5EREQAIJHc30w53lZZNNnHxcUhIyMDX331Fdzd3aFSqQDcX+fXxcXFkqERERHZDYt246empqK0tBT9+vWDn5+fdvvss88sGRYREdmh+y17U7rxLX0HdWfxbnwiIqIGYWI3vi2/emcVA/SIiIio/ljNe/ZERET1iTPoERER2Tkxj8ZnNz4REZGdY8ueiIhEQSqVQCqte/NcMOFYS2OyJyIiUWA3PhEREdkttuyJiEgUOBqfiIjIzom5G5/JnoiIREHMLXs+syciIrJzbNkTEZEoiLllbxfJXiMI0FjRojqaauuJpUb+mr9bOgQ9XkNWWDoEPaptUy0dgk1wbuRg6RBsgjUu9mXLCctUYn5mz258IiIiO2cXLXsiIqLHkcDEbnwbXuOWLXsiIhKFmm58UzZj/f7773j11Vfh5eUFFxcXhISE4Pjx49r9giBg3rx58PPzg4uLC8LDw3HhwgWdc9y4cQMxMTGQy+Xw8PDAuHHjUFZWZlQcTPZERET14ObNmwgLC0OjRo3wzTff4OzZs1i+fDmaNGmirbNs2TKsWrUKaWlpOHr0KFxdXREREYHy8nJtnZiYGJw5cwZZWVnIzMzEwYMHMXHiRKNiYTc+ERGJQkOPxn/33XcREBCADRs2aMuCgoK0/y0IApKTkzFnzhwMHToUAPDxxx/Dx8cH27ZtQ3R0NH7++Wfs2rULubm56N69OwBg9erVeOGFF/Dee+/B39/foFjYsiciIlEwVze+Wq3W2SoqKmq93tdff43u3bvjxRdfhLe3N7p06YJ169Zp9+fn50OlUiE8PFxbplAo0LNnT+Tk5AAAcnJy4OHhoU30ABAeHg6pVIqjR48afO9M9kREREYICAiAQqHQbklJSbXWu3TpElJTU9GmTRvs3r0bkyZNwptvvon09HQAgEqlAgD4+PjoHOfj46Pdp1Kp4O3trbPf0dERnp6e2jqGYDc+ERGJgrm68QsLCyGXy7XlMpms1voajQbdu3fH0qVLAQBdunTB6dOnkZaWhtjY2DrHURds2RMRkSiYqxtfLpfrbI9K9n5+fggODtYpa9++PQoKCgAAvr6+AICioiKdOkVFRdp9vr6+KC4u1tl/79493LhxQ1vHEEz2REQkCjUte1M2Y4SFheHcuXM6ZefPn0dgYCCA+4P1fH19kZ2drd2vVqtx9OhRKJVKAIBSqURJSQny8vK0dfbt2weNRoOePXsaHAu78YmIiOrBtGnT0Lt3byxduhQvvfQSjh07hrVr12Lt2rUA7v/xMXXqVCxevBht2rRBUFAQ5s6dC39/fwwbNgzA/Z6A559/HhMmTEBaWhqqqqoQHx+P6Ohog0fiA0z2REQkFibOjW/sBHo9evTA1q1bMXv2bCQmJiIoKAjJycmIiYnR1nn77bdx+/ZtTJw4ESUlJXj66aexa9cuODs7a+ts3rwZ8fHxGDBgAKRSKaKiorBq1SqjYmGyJyIiUbDEqneDBw/G4MGD//SciYmJSExMfGQdT09PZGRkGH3tB/GZPRERkZ1jy56IiERBzEvcMtkTEZEoWKIb31qwG5+IiMjOsWVPRESiwG58IiIiO8dufCIiIrJbbNkTEZEoiLllz2RPRESiwGf2REREdk7MLXs+syciIrJzbNkTEZEosBufiIjIzrEbn4iIiOwWW/ZERCQKEpjYjW+2SBoekz0REYmCVCKB1IRsb8qxlsZufCIiIjvHlj0REYkCR+MTERHZOTGPxmeyJyIiUZBK7m+mHG+r+MyeiIjIzrFlT0RE4iAxsSteTC379PR07NixQ/v57bffhoeHB3r37o3Lly+bNTgiIiJzqRmgZ8pmq4xu2S9duhSpqakAgJycHKSkpGDlypXIzMzEtGnT8OWXX5o9yMdxdJDC0YFPJP5Mye1KS4eg5+rWqZYOQU+z0Z9YOgQ91zbHWjoEPdUawdIh2ARbHtBF9sXoZF9YWIjWrVsDALZt24aoqChMnDgRYWFh6Nevn7njIyIiMgvJf/+ZcrytMro57ObmhuvXrwMA9uzZg+eeew4A4OzsjLt375o3OiIiIjOpGY1vymarjG7ZP/fccxg/fjy6dOmC8+fP44UXXgAAnDlzBi1atDB3fERERGQio1v2KSkpUCqVuHbtGr744gt4eXkBAPLy8vDyyy+bPUAiIiJzqJlUx5TNVhndsvfw8MAHH3ygV75w4UKzBERERFQfOF3uY/z0008Gn7BTp051DoaIiIjMz6BkHxoaColEAkGo/XWbmn0SiQTV1dVmDZCIiMgcuMTtY+Tn5+PSpUvIz8+vdavZd+nSpfqOl4iIqE4aelKdBQsW6D3zb9eunXZ/eXk54uLi4OXlBTc3N0RFRaGoqEjnHAUFBYiMjETjxo3h7e2NGTNm4N69e0bfu0Et+8DAQKNPTEREZE0ssepdhw4dsHfvXu1nR8f/pd1p06Zhx44d2LJlCxQKBeLj4zF8+HB8//33AIDq6mpERkbC19cXhw8fxtWrVzFq1Cg0atQIS5cuNSqOOk07t2nTJoSFhcHf3187RW5ycjK++uqrupyOiIjILjk6OsLX11e7PfHEEwCA0tJSrF+/HitWrMCzzz6Lbt26YcOGDTh8+DCOHDkC4P5cNmfPnsUnn3yC0NBQDBo0CIsWLUJKSgoqK42bFdXoZJ+amoqEhAS88MILKCkp0T6j9/DwQHJysrGnIyIiahDm6sZXq9U6W0VFxSOveeHCBfj7+6Nly5aIiYlBQUEBgPuvq1dVVSE8PFxbt127dmjevDlycnIA3J+SPiQkBD4+Pto6ERERUKvVOHPmjFH3bnSyX716NdatW4d//vOfcHBw0JZ3794dp06dMvZ0REREDaJmgJ4pGwAEBARAoVBot6SkpFqv17NnT2zcuBG7du1Camoq8vPz8cwzz+DWrVtQqVRwcnKCh4eHzjE+Pj5QqVQAAJVKpZPoa/bX7DOG0e/Z5+fno0uXLnrlMpkMt2/fNvZ0RERENqWwsBByuVz7WSaT1Vpv0KBB2v/u1KkTevbsicDAQHz++edwcXGp9zgfZHTLPigoCCdPntQr37VrF9q3b1/nQN555x1IJBJMnTq1zucgIiJ6FIkZNgCQy+U626OS/cM8PDzw5JNP4uLFi/D19UVlZSVKSkp06hQVFcHX1xcA4Ovrqzc6v+ZzTR1DGZ3sExISEBcXh88++wyCIODYsWNYsmQJZs+ejbffftvY0wEAcnNz8eGHH3JCHiIiqjeWni63rKwMv/76K/z8/NCtWzc0atQI2dnZ2v3nzp1DQUEBlEolAECpVOLUqVMoLi7W1snKyoJcLkdwcLBR1za6G3/8+PFwcXHBnDlzcOfOHbzyyivw9/fH+++/j+joaGNPh7KyMsTExGDdunVYvHix0ccTERFZo+nTp2PIkCEIDAzElStXMH/+fDg4OODll1+GQqHAuHHjkJCQAE9PT8jlckyePBlKpRK9evUCAAwcOBDBwcEYOXIkli1bBpVKhTlz5iAuLs7g3oQaRid7AIiJiUFMTAzu3LmDsrIyeHt71+U0AIC4uDhERkYiPDz8scm+oqJCZ9SjWq2u83WJiEhcTF2m1thj/+///g8vv/wyrl+/jqZNm+Lpp5/GkSNH0LRpUwDAypUrIZVKERUVhYqKCkRERGDNmjXa4x0cHJCZmYlJkyZBqVTC1dUVsbGxSExMNDr2OiV7ACguLsa5c+cA3O8aqQneGJ9++ilOnDiB3Nxcg+onJSVxwR0iIqqThp5U59NPP/3T/c7OzkhJSUFKSsoj6wQGBmLnzp1GXbc2Rj+zv3XrFkaOHAl/f3/07dsXffv2hb+/P1599VWUlpYafJ7CwkJMmTIFmzdvhrOzs0HHzJ49G6WlpdqtsLDQ2PCJiIhEx+hkP378eBw9ehQ7duxASUkJSkpKkJmZiePHj+O1114z+Dx5eXkoLi5G165d4ejoCEdHRxw4cACrVq2Co6NjrQvqyGQyvVGQREREhmqoefGtjdHd+JmZmdi9ezeefvppbVlERATWrVuH559/3uDzDBgwQG8SnjFjxqBdu3aYOXOmzoQ9REREprLE3PjWwuhk7+XlBYVCoVeuUCjQpEkTg8/j7u6Ojh076pS5urrCy8tLr5yIiMhUDT1Az5oY3Y0/Z84cJCQk6EzVp1KpMGPGDMydO9eswREREZHpDGrZd+nSRaf74sKFC2jevDmaN28O4P56uzKZDNeuXTPquf3D9u/fX+djiYiI/gy78R9j2LBh9RwGERFR/Xpwytu6Hm+rDEr28+fPr+84iIiIqJ7UeVIdIiIiW/LgMrV1Pd5WGZ3sq6ursXLlSnz++ecoKChAZWWlzv4bN26YLTgiIiJzMfV9eRvO9caPxl+4cCFWrFiBESNGoLS0FAkJCRg+fDikUikWLFhQDyESERGRKYxO9ps3b8a6devw1ltvwdHRES+//DL+3//7f5g3bx6OHDlSHzESERGZzNJL3FqS0clepVIhJCQEAODm5qadD3/w4MHYsWOHeaMjIiIyE1OmyrX1KXONTvbNmjXD1atXAQCtWrXCnj17AAC5ublGr69LRERE9c/oZP+3v/0N2dnZAIDJkydj7ty5aNOmDUaNGoWxY8eaPUAiIiJzqBmNb8pmq4wejf/OO+9o/3vEiBEIDAzE4cOH0aZNGwwZMsSswREREZkLR+OboFevXkhISEDPnj2xdOlSc8RERERkdhygZwZXr17lQjhERERWiDPoiYSHq5OlQ7AJ1zbHWjoEPU2UCZYOQc/NnBWWDkFPtUawdAh6HKxwTdR71RpLh6CjIeORwrQWrtlaxxbAZE9ERKIg5lXvbPkPFSIiIjKAwS37hIQ/70q8du2aycEQERHVF4kEMOXJig037A1P9j/88MNj6/Tp08ekYIiIiOqL1MRkb4VDMAxmcLL/9ttv6zMOIiIiqiccoEdERKIg5gF6TPZERCQKYu7G52h8IiIiO8eWPRERiYKY58ZnsiciIlEwdeU6W171rk7d+N999x1effVVKJVK/P777wCATZs24dChQ2YNjoiIyFykZthsldGxf/HFF4iIiICLiwt++OEHVFRUAABKS0u56h0REZEVMjrZL168GGlpaVi3bh0aNWqkLQ8LC8OJEyfMGhwREZG51DyzN2Wrq3feeQcSiQRTp07VlpWXlyMuLg5eXl5wc3NDVFQUioqKdI4rKChAZGQkGjduDG9vb8yYMQP37t0z+vpGJ/tz587VOlOeQqFASUmJ0QEQERE1BCkk2uf2ddpQt2yfm5uLDz/8EJ06ddIpnzZtGrZv344tW7bgwIEDuHLlCoYPH67dX11djcjISFRWVuLw4cNIT0/Hxo0bMW/evDrcu5F8fX1x8eJFvfJDhw6hZcuWRgdARERkr8rKyhATE4N169ahSZMm2vLS0lKsX78eK1aswLPPPotu3bphw4YNOHz4MI4cOQIA2LNnD86ePYtPPvkEoaGhGDRoEBYtWoSUlBRUVlYaFYfRyX7ChAmYMmUKjh49ColEgitXrmDz5s2YPn06Jk2aZOzpiIiIGoS5uvHVarXOVjN2rTZxcXGIjIxEeHi4TnleXh6qqqp0ytu1a4fmzZsjJycHAJCTk4OQkBD4+Pho60RERECtVuPMmTNG3bvRr97NmjULGo0GAwYMwJ07d9CnTx/IZDJMnz4dkydPNvZ0REREDcJcM+gFBATolM+fPx8LFizQq//pp5/ixIkTyM3N1dunUqng5OQEDw8PnXIfHx+oVCptnQcTfc3+mn3GMDrZSyQS/POf/8SMGTNw8eJFlJWVITg4GG5ubsaeioiIyOYUFhZCLpdrP8tkslrrTJkyBVlZWXB2dm7I8GpV50l1nJycEBwcbM5YiIiI6s399exNWQjn/v/K5XKdZF+bvLw8FBcXo2vXrtqy6upqHDx4EB988AF2796NyspKlJSU6LTui4qK4OvrC+D+GLljx47pnLdmtH5NHUMZnez79+//pyv/7Nu3z9hTEhER1buGnC53wIABOHXqlE7ZmDFj0K5dO8ycORMBAQFo1KgRsrOzERUVBeD+224FBQVQKpUAAKVSiSVLlqC4uBje3t4AgKysLMjlcqMb20Yn+9DQUJ3PVVVVOHnyJE6fPo3Y2FhjT0dERGR33N3d0bFjR50yV1dXeHl5acvHjRuHhIQEeHp6Qi6XY/LkyVAqlejVqxcAYODAgQgODsbIkSOxbNkyqFQqzJkzB3FxcbU+OvgzRif7lStX1lq+YMEClJWVGXs6IiKiBmFtS9yuXLkSUqkUUVFRqKioQEREBNasWaPd7+DggMzMTEyaNAlKpRKurq6IjY1FYmKi0deSCIIgmCPoixcv4qmnnsKNGzfMcTqDqNVqKBQKFF0vfezzEyJb1USZYOkQ9NzMWWHpEPRUa8zyq8ysHKxwAfR71RpLh6BDrVbjL95NUFpaf7/Ha3LF3K9+gLOre53PU377FhYN7VKvsdYXs616l5OTYxUjDomIiGpjbS37hmR0sn9wKj8AEAQBV69exfHjxzF37lyzBUZERETmYXSyVygUOp+lUinatm2LxMREDBw40GyBERERmRNb9gaqrq7GmDFjEBISojPHLxERkbWTSCR/+uq4IcfbKqPmxndwcMDAgQO5uh0REZENMXohnI4dO+LSpUv1EQsREVG9qenGN2WzVUYn+8WLF2P69OnIzMzE1atX9Vb/ISIiskbmWvXOFhn8zD4xMRFvvfUWXnjhBQDAX//6V53nF4IgQCKRoLq62vxREhERUZ0ZnOwXLlyI119/Hd9++219xkNERFQvpBKJSQvhmHKspRmc7Gsm2uvbt2+9BUNERFRfxPzqnVHP7G35tQMiIiKxMuo9+yeffPKxCb8h58YnIiIymKmD7Gy4vWtUsl+4cKHeDHpERES2QAoJpCZkbFOOtTSjkn10dDS8vb3rK5Y6U9+pguBYZekwtBSNG1k6BD03yiotHYIeV5mDpUPQU1Z+z9Ih6LHGFeY6/WOXpUPQ89PS5y0dgh4zLSpqVo4ORr9xXa8aMh5TX5+z5SfZBn/LfF5PRERkm4wejU9ERGSLxDwa3+Bkr9Fo6jMOIiKieiXm9+yt6+ENERERmZ3R69kTERHZIjEP0GOyJyIiUZDCxG58G371jt34REREdo4teyIiEgV24xMREdk5KUzrzrblrnBbjp2IiIgMwJY9ERGJgkQiMWk2WFueSZbJnoiIREEC0xaus91Uz2RPREQiwRn0LOj333/Hq6++Ci8vL7i4uCAkJATHjx+3dFhERER2w6It+5s3byIsLAz9+/fHN998g6ZNm+LChQto0qSJJcMiIiI7Zbttc9NYNNm/++67CAgIwIYNG7RlQUFBFoyIiIjslZjfs7doN/7XX3+N7t2748UXX4S3tze6dOmCdevWPbJ+RUUF1Gq1zkZERER/zqLJ/tKlS0hNTUWbNm2we/duTJo0CW+++SbS09NrrZ+UlASFQqHdAgICGjhiIiKyVTWv3pmyGSM1NRWdOnWCXC6HXC6HUqnEN998o91fXl6OuLg4eHl5wc3NDVFRUSgqKtI5R0FBASIjI9G4cWN4e3tjxowZuHfvntH3btFkr9Fo0LVrVyxduhRdunTBxIkTMWHCBKSlpdVaf/bs2SgtLdVuhYWFDRwxERHZKqkZNmM0a9YM77zzDvLy8nD8+HE8++yzGDp0KM6cOQMAmDZtGrZv344tW7bgwIEDuHLlCoYPH649vrq6GpGRkaisrMThw4eRnp6OjRs3Yt68eUbfu0Wf2fv5+SE4OFinrH379vjiiy9qrS+TySCTyRoiNCIiIpMMGTJE5/OSJUuQmpqKI0eOoFmzZli/fj0yMjLw7LPPAgA2bNiA9u3b48iRI+jVqxf27NmDs2fPYu/evfDx8UFoaCgWLVqEmTNnYsGCBXBycjI4Fou27MPCwnDu3DmdsvPnzyMwMNBCERERkb0yVzf+w2PHKioqHnvt6upqfPrpp7h9+zaUSiXy8vJQVVWF8PBwbZ127dqhefPmyMnJAQDk5OQgJCQEPj4+2joRERFQq9Xa3gFDWTTZT5s2DUeOHMHSpUtx8eJFZGRkYO3atYiLi7NkWEREZIckZtgAICAgQGf8WFJS0iOveerUKbi5uUEmk+H111/H1q1bERwcDJVKBScnJ3h4eOjU9/HxgUqlAgCoVCqdRF+zv2afMSzajd+jRw9s3boVs2fPRmJiIoKCgpCcnIyYmBhLhkVERPRIhYWFkMvl2s9/9ni5bdu2OHnyJEpLS/Gf//wHsbGxOHDgQEOEqcPi0+UOHjwYgwcPtnQYRERk58y1EE7N6HpDODk5oXXr1gCAbt26ITc3F++//z5GjBiByspKlJSU6LTui4qK4OvrCwDw9fXFsWPHdM5XM1q/po6hLD5dLhERUUNo6NH4tdFoNKioqEC3bt3QqFEjZGdna/edO3cOBQUFUCqVAAClUolTp06huLhYWycrKwtyuVxvcPvjWLxlT0RE1BAaeonb2bNnY9CgQWjevDlu3bqFjIwM7N+/H7t374ZCocC4ceOQkJAAT09PyOVyTJ48GUqlEr169QIADBw4EMHBwRg5ciSWLVsGlUqFOXPmIC4uzug305jsiYiI6kFxcTFGjRqFq1evQqFQoFOnTti9ezeee+45AMDKlSshlUoRFRWFiooKREREYM2aNdrjHRwckJmZiUmTJkGpVMLV1RWxsbFITEw0OhYmeyIiEoWGXs9+/fr1f7rf2dkZKSkpSElJeWSdwMBA7Ny508gr62OyJyIiUeBCOERERGS32LInIiJRkEICqQkd+aYca2lM9kREJArsxiciIiK7xZY9ERGJguS//0w53lYx2RMRkSiwG5+IiIjsll207OWNG0HeuJGlw7Bqnm5Olg7BJsgaOVg6BD33qjWWDkHPT0uft3QIepr0mW3pEPTcPPjopU8tpfKedf08NWQ8EhNH47Mbn4iIyMqJuRufyZ6IiERBzMmez+yJiIjsHFv2REQkCnz1joiIyM5JJfc3U463VezGJyIisnNs2RMRkSiwG5+IiMjOcTQ+ERER2S227ImISBQkMK0r3oYb9kz2REQkDhyNT0RERHaLLXsiIhIFjsYnIiKyc2Iejc9kT0REoiCBaYPsbDjX85k9ERGRvWPLnoiIREEKCaQm9MVLbbhtz2RPRESiwG58IiIislts2RMRkTiIuGnPZE9ERKIg5vfs2Y1PRERk55jsiYhIHCT/m1inLpuxDfukpCT06NED7u7u8Pb2xrBhw3Du3DmdOuXl5YiLi4OXlxfc3NwQFRWFoqIinToFBQWIjIxE48aN4e3tjRkzZuDevXtGxcJkT0REoiAxw2aMAwcOIC4uDkeOHEFWVhaqqqowcOBA3L59W1tn2rRp2L59O7Zs2YIDBw7gypUrGD58uHZ/dXU1IiMjUVlZicOHDyM9PR0bN27EvHnzjIqFz+yJiIjqwa5du3Q+b9y4Ed7e3sjLy0OfPn1QWlqK9evXIyMjA88++ywAYMOGDWjfvj2OHDmCXr16Yc+ePTh79iz27t0LHx8fhIaGYtGiRZg5cyYWLFgAJycng2Jhy56IiMTBTE17tVqts1VUVBh0+dLSUgCAp6cnACAvLw9VVVUIDw/X1mnXrh2aN2+OnJwcAEBOTg5CQkLg4+OjrRMREQG1Wo0zZ84YfOtM9kREJAoSM/wDgICAACgUCu2WlJT02GtrNBpMnToVYWFh6NixIwBApVLByckJHh4eOnV9fHygUqm0dR5M9DX7a/YZit34REQkCuZa9a6wsBByuVxbLpPJHntsXFwcTp8+jUOHDtU9ABOwZU9ERGQEuVyusz0u2cfHxyMzMxPffvstmjVrpi339fVFZWUlSkpKdOoXFRXB19dXW+fh0fk1n2vqGILJnoiIRKGhR+MLgoD4+Hhs3boV+/btQ1BQkM7+bt26oVGjRsjOztaWnTt3DgUFBVAqlQAApVKJU6dOobi4WFsnKysLcrkcwcHBBsfCbnwiIhKHBp4uNy4uDhkZGfjqq6/g7u6ufcauUCjg4uIChUKBcePGISEhAZ6enpDL5Zg8eTKUSiV69eoFABg4cCCCg4MxcuRILFu2DCqVCnPmzEFcXJxBjw9qMNkTERHVg9TUVABAv379dMo3bNiA0aNHAwBWrlwJqVSKqKgoVFRUICIiAmvWrNHWdXBwQGZmJiZNmgSlUglXV1fExsYiMTHRqFiY7ImISBQaem58QRAeW8fZ2RkpKSlISUl5ZJ3AwEDs3LnTqGs/jMmeiIhEwVyj8W0RB+gRERHZObbsiYhIFES8nD2TPdGDDHnG1tAcHayvA06jsb7v6ebBx89i1tCaPDXZ0iHouXlstaVD0OHk2IA/3yLO9tb3W4SIiIjMii17IiIShYYejW9NmOyJiEgUxDwan8meiIhEQcSP7PnMnoiIyN6xZU9EROIg4qY9kz0REYmCmAfosRufiIjIzrFlT0REosDR+ERERHZOxI/s2Y1PRERk79iyJyIicRBx057JnoiIRIGj8S2kuroac+fORVBQEFxcXNCqVSssWrTIKlceIyIislUWbdm/++67SE1NRXp6Ojp06IDjx49jzJgxUCgUePPNNy0ZGhER2RmOxreQw4cPY+jQoYiMjAQAtGjRAv/+979x7NgxS4ZFRER2SMSP7C3bjd+7d29kZ2fj/PnzAIAff/wRhw4dwqBBg2qtX1FRAbVarbMREREZRGKGzUZZtGU/a9YsqNVqtGvXDg4ODqiursaSJUsQExNTa/2kpCQsXLiwgaMkIiKybRZt2X/++efYvHkzMjIycOLECaSnp+O9995Denp6rfVnz56N0tJS7VZYWNjAERMRka2SmOGfrbJoy37GjBmYNWsWoqOjAQAhISG4fPkykpKSEBsbq1dfJpNBJpM1dJhERGQPTBygZ8O53rIt+zt37kAq1Q3BwcEBGo3GQhERERHZH4u27IcMGYIlS5agefPm6NChA3744QesWLECY8eOtWRYRERkh8Q8Gt+iyX716tWYO3cu3njjDRQXF8Pf3x+vvfYa5s2bZ8mwiIjIHok421s02bu7uyM5ORnJycmWDIOIiMiucW58IiISBTHPjc9kT0REoiDm6XK5nj0REVE9OHjwIIYMGQJ/f39IJBJs27ZNZ78gCJg3bx78/Pzg4uKC8PBwXLhwQafOjRs3EBMTA7lcDg8PD4wbNw5lZWVGx8JkT0REotDQs+Xevn0bnTt3RkpKSq37ly1bhlWrViEtLQ1Hjx6Fq6srIiIiUF5erq0TExODM2fOICsrC5mZmTh48CAmTpxoZCTsxiciIrFo4NH4gwYNeuRaL4IgIDk5GXPmzMHQoUMBAB9//DF8fHywbds2REdH4+eff8auXbuQm5uL7t27A7j/FtsLL7yA9957D/7+/gbHwpY9ERGJgjVNl5ufnw+VSoXw8HBtmUKhQM+ePZGTkwMAyMnJgYeHhzbRA0B4eDikUimOHj1q1PXYsiciIjLCwyuu1mUqd5VKBQDw8fHRKffx8dHuU6lU8Pb21tnv6OgIT09PbR1DsWVPRESiIMH/RuTXafvveQICAqBQKLRbUlKSJW/LIGzZExGRKJjrkX1hYSHkcrm2vC4LtPn6+gIAioqK4Ofnpy0vKipCaGiotk5xcbHOcffu3cONGze0xxuKLXsiIiIjyOVyna0uyT4oKAi+vr7Izs7WlqnVahw9ehRKpRIAoFQqUVJSgry8PG2dffv2QaPRoGfPnkZdjy17IiIShYaeVKesrAwXL17Ufs7Pz8fJkyfh6emJ5s2bY+rUqVi8eDHatGmDoKAgzJ07F/7+/hg2bBgAoH379nj++ecxYcIEpKWloaqqCvHx8YiOjjZqJD7AZE9ERKLRsO/eHT9+HP3799d+TkhIAADExsZi48aNePvtt3H79m1MnDgRJSUlePrpp7Fr1y44Oztrj9m8eTPi4+MxYMAASKVSREVFYdWqVcZHLgiCYPRRVkKtVkOhUKDoeqnO8xMie2KN/xeVWOG8oZX3NJYOQY+To/U9KW3SZ7alQ9Ah3KtARe5KlJbW3+/xmlxx9rdrcDfhGrfUagS3aFqvsdYXtuyJiEgUxDw3PpM9ERGJgoiXs+dofCIiInvHlj0REYkCu/GJiIjsnKnz25tzbvyGxmRPRETiIOKH9nxmT0REZOfYsiciIlEQccOeyZ6IiMRBzAP02I1PRERk59iyJyIiUeBofCIiInsn4of27MYnIiKyc2zZExGRKIi4Yc9kT0RE4sDR+ERERGS32LInIiKRMG00vi135DPZExGRKLAbn4iIiOwWkz0REZGdYzc+ERGJgpi78ZnsiYhIFMQ8XS678YmIiOwcW/ZERCQK7MYnIiKyc2KeLpfd+ERERHaOLXsiIhIHETftmeyJiEgUOBqfiIiI7BZb9kREJAocjU9ERGTnRPzInt34REQkEhIzbHWQkpKCFi1awNnZGT179sSxY8dMu486YLInIiKqJ5999hkSEhIwf/58nDhxAp07d0ZERASKi4sbNA4meyIiEgWJGf4Za8WKFZgwYQLGjBmD4OBgpKWloXHjxvjoo4/q4Q4fjcmeiIhEoWaAnimbMSorK5GXl4fw8HBtmVQqRXh4OHJycsx8d3/OpgfoCYIAALilVls4EqL6U/Nzbk0kVjgsufKextIh6HFytL72lHCvwtIh6BCq78fTED/nahNzRc3xD59HJpNBJpPp1f/jjz9QXV0NHx8fnXIfHx/88ssvJsViLJtO9rdu3QIAtA4KsHAkRERkilu3bkGhUNTLuZ2cnODr64s2ZsgVbm5uCAjQPc/8+fOxYMECk89dn2w62fv7+6OwsBDu7u4mtzTUajUCAgJQWFgIuVxupgjtD7+nx+N3ZBh+T4ax9+9JEATcunUL/v7+9XYNZ2dn5Ofno7Ky0uRzCYKgl29qa9UDwBNPPAEHBwcUFRXplBcVFcHX19fkWIxh08leKpWiWbNmZj2nXC63y/9DmRu/p8fjd2QYfk+Gsefvqb5a9A9ydnaGs7NzvV/nQU5OTujWrRuys7MxbNgwAIBGo0F2djbi4+MbNBabTvZERETWLCEhAbGxsejevTueeuopJCcn4/bt2xgzZkyDxsFkT0REVE9GjBiBa9euYd68eVCpVAgNDcWuXbv0Bu3VNyb7/5LJZJg/f/4jn73QffyeHo/fkWH4PRmG35Pti4+Pb/Bu+4dJBGt8r4eIiIjMxvpeAiUiIiKzYrInIiKyc0z2REREdo7JnoiIyM4x2cM61hq2ZklJSejRowfc3d3h7e2NYcOG4dy5c5YOy+q98847kEgkmDp1qqVDsTq///47Xn31VXh5ecHFxQUhISE4fvy4pcOyKtXV1Zg7dy6CgoLg4uKCVq1aYdGiRVa5VgJZP9Ene2tZa9iaHThwAHFxcThy5AiysrJQVVWFgQMH4vbt25YOzWrl5ubiww8/RKdOnSwditW5efMmwsLC0KhRI3zzzTc4e/Ysli9fjiZNmlg6NKvy7rvvIjU1FR988AF+/vlnvPvuu1i2bBlWr15t6dDIBon+1buePXuiR48e+OCDDwDcn8owICAAkydPxqxZsywcnXW6du0avL29ceDAAfTp08fS4VidsrIydO3aFWvWrMHixYsRGhqK5ORkS4dlNWbNmoXvv/8e3333naVDsWqDBw+Gj48P1q9fry2LioqCi4sLPvnkEwtGRrZI1C17a1pr2JaUlpYCADw9PS0ciXWKi4tDZGSkzs8V/c/XX3+N7t2748UXX4S3tze6dOmCdevWWTosq9O7d29kZ2fj/PnzAIAff/wRhw4dwqBBgywcGdkiUc+gZ01rDdsKjUaDqVOnIiwsDB07drR0OFbn008/xYkTJ5Cbm2vpUKzWpUuXkJqaioSEBPzjH/9Abm4u3nzzTTg5OSE2NtbS4VmNWbNmQa1Wo127dnBwcEB1dTWWLFmCmJgYS4dGNkjUyZ6MFxcXh9OnT+PQoUOWDsXqFBYWYsqUKcjKymrw1bVsiUajQffu3bF06VIAQJcuXXD69GmkpaUx2T/g888/x+bNm5GRkYEOHTrg5MmTmDp1Kvz9/fk9kdFEneytaa1hWxAfH4/MzEwcPHjQ7EsL24O8vDwUFxeja9eu2rLq6mocPHgQH3zwASoqKuDg4GDBCK2Dn58fgoODdcrat2+PL774wkIRWacZM2Zg1qxZiI6OBgCEhITg8uXLSEpKYrIno4n6mf2Daw3XqFlrWKlUWjAy6yIIAuLj47F161bs27cPQUFBlg7JKg0YMACnTp3CyZMntVv37t0RExODkydPMtH/V1hYmN6rm+fPn0dgYKCFIrJOd+7cgVSq+yvawcEBGo3GQhGRLRN1yx6wnrWGrVlcXBwyMjLw1Vdfwd3dHSqVCgCgUCjg4uJi4eish7u7u944BldXV3h5eXF8wwOmTZuG3r17Y+nSpXjppZdw7NgxrF27FmvXrrV0aFZlyJAhWLJkCZo3b44OHTrghx9+wIoVKzB27FhLh0a2SCBh9erVQvPmzQUnJyfhqaeeEo4cOWLpkKwKgFq3DRs2WDo0q9e3b19hypQplg7D6mzfvl3o2LGjIJPJhHbt2glr1661dEhWR61WC1OmTBGaN28uODs7Cy1bthT++c9/ChUVFZYOjWyQ6N+zJyIisneifmZPREQkBkz2REREdo7JnoiIyM4x2RMREdk5JnsiIiI7x2RPRERk55jsiYiI7ByTPZERRo8ejWHDhmk/9+vXD1OnTm3wOPbv3w+JRIKSkpJ6u8bD91oXDREnET0ekz3ZvNGjR0MikUAikcDJyQmtW7dGYmIi7t27V+/X/vLLL7Fo0SKD6jZ04mvRogWSk5Mb5FpEZN1EPzc+2Yfnn38eGzZsQEVFBXbu3Im4uDg0atQIs2fP1qtbWVkJJycns1zX09PTLOchIqpPbNmTXZDJZPD19UVgYCAmTZqE8PBwfP311wD+1x29ZMkS+Pv7o23btgDurz//0ksvwcPDA56enhg6dCh+++037Tmrq6uRkJAADw8PeHl54e2338bDs0s/3I1fUVGBmTNnIiAgADKZDK1bt8b69evx22+/oX///gCAJk2aQCKRYPTo0QDur7SYlJSEoKAguLi4oHPnzvjPf/6jc52dO3fiySefhIuLC/r3768TZ11UV1dj3Lhx2mu2bdsW77//fq11Fy5ciKZNm0Iul+P1119HZWWldp8hsT/o8uXLGDJkCJo0aQJXV1d06NABO3fuNOleiOjx2LInu+Ti4oLr169rP2dnZ0MulyMrKwsAUFVVhYiICCiVSnz33XdwdHTE4sWL8fzzz+Onn36Ck5MTli9fjo0bN+Kjjz5C+/btsXz5cmzduhXPPvvsI687atQo5OTkYNWqVejcuTPy8/Pxxx9/ICAgAF988QWioqJw7tw5yOVy7YqBSUlJ+OSTT5CWloY2bdrg4MGDePXVV9G0aVP07dsXhYWFGD58OOLi4jBx4kQcP34cb731lknfj0ajQbNmzbBlyxZ4eXnh8OHDmDhxIvz8/PDSSy/pfG/Ozs7Yv38/fvvtN4wZMwZeXl5YsmSJQbE/LC4uDpWVlTh48CBcXV1x9uxZuLm5mXQvRGQACy/EQ2Sy2NhYYejQoYIgCIJGoxGysrIEmUwmTJ8+Xbvfx8dHZ7WwTZs2CW3bthU0Go22rKKiQnBxcRF2794tCIIg+Pn5CcuWLdPur6qqEpo1a6a9liDormp37tw5AYCQlZVVa5zffvutAEC4efOmtqy8vFxo3LixcPjwYZ2648aNE15++WVBEARh9uzZQnBwsM7+mTNn6p3rYYGBgcLKlSsfuf9hcXFxQlRUlPZzbGys4OnpKdy+fVtblpqaKri5uQnV1dUGxf7wPYeEhAgLFiwwOCYiMg+27MkuZGZmws3NDVVVVdBoNHjllVewYMEC7f6QkBCd5/Q//vgjLl68CHd3d53zlJeX49dff0VpaSmuXr2Knj17avc5Ojqie/fuel35NU6ePAkHB4daW7SPcvHiRdy5cwfPPfecTnllZSW6dOkCAPj555914gAApVJp8DUeJSUlBR999BEKCgpw9+5dVFZWIjQ0VKdO586d0bhxY53rlpWVobCwEGVlZY+N/WFvvvkmJk2ahD179iA8PBxRUVHo1KmTyfdCRH+OyZ7sQv/+/ZGamgonJyf4+/vD0VH3R9vV1VXnc1lZGbp164bNmzfrnatp06Z1iqGmW94YZWVlAIAdO3bgL3/5i84+mUxWpzgM8emnn2L69OlYvnw5lEol3N3d8a9//QtHjx41+Bx1iX38+PGIiIjAjh07sGfPHiQlJWH58uWYPHly3W+GiB6LyZ7sgqurK1q3bm1w/a5du+Kzzz6Dt7c35HJ5rXX8/Pxw9OhR9OnTBwBw79495OXloWvXrrXWDwkJgUajwYEDBxAeHq63v6Znobq6WlsWHBwMmUyGgoKCR/YItG/fXjvYsMaRI0cef5N/4vvvv0fv3r3xxhtvaMt+/fVXvXo//vgj7t69q/1D5siRI3Bzc0NAQAA8PT0fG3ttAgIC8Prrr+P111/H7NmzsW7dOiZ7onrG0fgkSjExMXjiiScwdOhQfPfdd8jPz8f+/fvx5ptv4v/+7/8AAFOmTME777yDbdu24ZdffsEbb7zxp+/It2jRArGxsRg7diy2bdumPefnn38OAAgMDIREIkFmZiauXbuGsrIyuLu7Y/r06Zg2bRrS09Px66+/4sSJE1i9ejXS09MBAK+//jouXLiAGTNm4Ny5c8jIyMDGjRsNus/ff/8dJ0+e1Nlu3ryJNm3a4Pjx49i9ezfOnz+PuXPnIjc3V+/4yspKjBs3DmfPnsXOnTsxf/58xMfHQyqVGhT7w6ZOnYrdu3cjPz8fJ06cwLfffov27dsbdC9EZAJLDxogMtWDA/SM2X/16lVh1KhRwhNPPCHIZDKhZcuWwoQJE4TS0lJBEO4PyJsyZYogl8sFDw8PISEhQRg1atQjB+gJgiDcvXtXmDZtmuDn5yc4OTkJrVu3Fj766CPt/sTERMHX11eQSCRCbGysIAj3BxUmJycLbdu2FRo1aiQ0bdpUiIiIEA4cOKA9bvv27ULr1q0FmUwmPPPMM8JHH31k0AA9AHrbpk2bhPLycmH06NGCQqEQPDw8hEmTJgmzZs0SOnfurPe9zZs3T/Dy8hLc3NyECRMmCOXl5do6j4v94QF68fHxQqtWrQSZTCY0bdpUGDlypPDHH3888h6IyDwkgvCI0UZERERkF9iNT0REZOeY7ImIiOwckz0REZGdY7InIiKyc0z2REREdo7JnoiIyM4x2RMREdk5JnsiIiI7x2RPRERk55jsiYiI7ByTPRERkZ1jsiciIrJz/x87SqA+V9IFsQAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Certainly! Let's define and train multi-layer perceptron (MLP) models with 2 and 3 layers, and then compare their performance with the baseline single-layer model.\n",
        "\n",
        "### Define MLP Models with 2 and 3 Layers:\n",
        "We'll define MLP models with additional hidden layers to see how the performance changes with the increase in the number of layers.\n",
        "\n",
        "#### Two-Layer MLP:\n",
        "The two-layer MLP will consist of an input layer, two hidden layers, and an output layer.\n",
        "\n",
        "#### Three-Layer MLP:\n",
        "The three-layer MLP will consist of an input layer, three hidden layers, and an output layer.\n",
        "\n",
        "Now, let's define and train these models:\n",
        "\n",
        "```python\n",
        "# Define a two-layer MLP model\n",
        "def create_two_layer_mlp(input_shape, num_classes):\n",
        "    model = Sequential()\n",
        "    # Add first hidden layer with ReLU activation\n",
        "    model.add(Dense(128, activation='relu', input_shape=input_shape))\n",
        "    # Add second hidden layer with ReLU activation\n",
        "    model.add(Dense(64, activation='relu'))\n",
        "    # Add output layer with softmax activation for multi-class classification\n",
        "    model.add(Dense(num_classes, activation='softmax'))\n",
        "    return model\n",
        "\n",
        "# Define a three-layer MLP model\n",
        "def create_three_layer_mlp(input_shape, num_classes):\n",
        "    model = Sequential()\n",
        "    # Add first hidden layer with ReLU activation\n",
        "    model.add(Dense(128, activation='relu', input_shape=input_shape))\n",
        "    # Add second hidden layer with ReLU activation\n",
        "    model.add(Dense(64, activation='relu'))\n",
        "    # Add third hidden layer with ReLU activation\n",
        "    model.add(Dense(32, activation='relu'))\n",
        "    # Add output layer with softmax activation for multi-class classification\n",
        "    model.add(Dense(num_classes, activation='softmax'))\n",
        "    return model\n",
        "\n",
        "# Create two-layer MLP model\n",
        "two_layer_model = create_two_layer_mlp(input_shape, num_classes)\n",
        "\n",
        "# Create three-layer MLP model\n",
        "three_layer_model = create_three_layer_mlp(input_shape, num_classes)\n",
        "\n",
        "# Train two-layer MLP model\n",
        "two_layer_history = train_model(two_layer_model, x_train, y_train, x_val, y_val)\n",
        "\n",
        "# Train three-layer MLP model\n",
        "three_layer_history = train_model(three_layer_model, x_train, y_train, x_val, y_val)\n",
        "```\n",
        "\n",
        "### Model Comparison:\n",
        "Once the models are trained, we can compare their performance based on evaluation metrics such as accuracy, precision, recall, and confusion matrix.\n",
        "\n",
        "```python\n",
        "# Evaluate two-layer MLP model\n",
        "two_layer_accuracy, _, _, _ = evaluate_model(two_layer_model, x_test, y_test)\n",
        "\n",
        "# Evaluate three-layer MLP model\n",
        "three_layer_accuracy, _, _, _ = evaluate_model(three_layer_model, x_test, y_test)\n",
        "\n",
        "# Print model comparison\n",
        "print(\"Single-layer MLP Accuracy:\", accuracy)\n",
        "print(\"Two-layer MLP Accuracy:\", two_layer_accuracy)\n",
        "print(\"Three-layer MLP Accuracy:\", three_layer_accuracy)\n",
        "```\n",
        "\n",
        "Explanation:\n",
        "- We define two additional MLP models with two and three hidden layers using the previously defined functions.\n",
        "- We train both models using the training data and validation data.\n",
        "- After training, we evaluate the performance of each model using the testing data and compare their accuracies.\n",
        "- The comparison helps us understand how the addition of layers affects the model's performance."
      ],
      "metadata": {
        "id": "MrSWLuRbsI9g"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Define a two-layer MLP model\n",
        "def create_two_layer_mlp(input_shape, num_classes):\n",
        "    model = Sequential()\n",
        "    # Add first hidden layer with ReLU activation\n",
        "    model.add(Dense(128, activation='relu', input_shape=input_shape))\n",
        "    # Add second hidden layer with ReLU activation\n",
        "    model.add(Dense(64, activation='relu'))\n",
        "    # Add output layer with softmax activation for multi-class classification\n",
        "    model.add(Dense(num_classes, activation='softmax'))\n",
        "    return model\n",
        "\n",
        "# Define a three-layer MLP model\n",
        "def create_three_layer_mlp(input_shape, num_classes):\n",
        "    model = Sequential()\n",
        "    # Add first hidden layer with ReLU activation\n",
        "    model.add(Dense(128, activation='relu', input_shape=input_shape))\n",
        "    # Add second hidden layer with ReLU activation\n",
        "    model.add(Dense(64, activation='relu'))\n",
        "    # Add third hidden layer with ReLU activation\n",
        "    model.add(Dense(32, activation='relu'))\n",
        "    # Add output layer with softmax activation for multi-class classification\n",
        "    model.add(Dense(num_classes, activation='softmax'))\n",
        "    return model\n",
        "\n",
        "# Create two-layer MLP model\n",
        "two_layer_model = create_two_layer_mlp(input_shape, num_classes)\n",
        "\n",
        "# Create three-layer MLP model\n",
        "three_layer_model = create_three_layer_mlp(input_shape, num_classes)\n",
        "\n",
        "# Train two-layer MLP model\n",
        "two_layer_history = train_model(two_layer_model, x_train, y_train, x_val, y_val)\n",
        "\n",
        "# Train three-layer MLP model\n",
        "three_layer_history = train_model(three_layer_model, x_train, y_train, x_val, y_val)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "d9_kw6XQsJKj",
        "outputId": "2fd10e0c-a2ed-4b1f-a5da-f0ea7d1705b1"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "1688/1688 [==============================] - 7s 3ms/step - loss: 0.4978 - accuracy: 0.8214 - val_loss: 0.4006 - val_accuracy: 0.8560\n",
            "Epoch 2/10\n",
            "1688/1688 [==============================] - 6s 4ms/step - loss: 0.3729 - accuracy: 0.8630 - val_loss: 0.3825 - val_accuracy: 0.8570\n",
            "Epoch 3/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.3370 - accuracy: 0.8760 - val_loss: 0.3579 - val_accuracy: 0.8560\n",
            "Epoch 4/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.3103 - accuracy: 0.8836 - val_loss: 0.3344 - val_accuracy: 0.8760\n",
            "Epoch 5/10\n",
            "1688/1688 [==============================] - 6s 4ms/step - loss: 0.2954 - accuracy: 0.8912 - val_loss: 0.3371 - val_accuracy: 0.8720\n",
            "Epoch 6/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2799 - accuracy: 0.8947 - val_loss: 0.3409 - val_accuracy: 0.8730\n",
            "Epoch 7/10\n",
            "1688/1688 [==============================] - 6s 3ms/step - loss: 0.2680 - accuracy: 0.9004 - val_loss: 0.3517 - val_accuracy: 0.8640\n",
            "Epoch 8/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2560 - accuracy: 0.9038 - val_loss: 0.3244 - val_accuracy: 0.8780\n",
            "Epoch 9/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2472 - accuracy: 0.9076 - val_loss: 0.3527 - val_accuracy: 0.8770\n",
            "Epoch 10/10\n",
            "1688/1688 [==============================] - 6s 3ms/step - loss: 0.2368 - accuracy: 0.9111 - val_loss: 0.3263 - val_accuracy: 0.8830\n",
            "Epoch 1/10\n",
            "1688/1688 [==============================] - 7s 3ms/step - loss: 0.5242 - accuracy: 0.8151 - val_loss: 0.4453 - val_accuracy: 0.8400\n",
            "Epoch 2/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.3826 - accuracy: 0.8615 - val_loss: 0.3898 - val_accuracy: 0.8530\n",
            "Epoch 3/10\n",
            "1688/1688 [==============================] - 6s 4ms/step - loss: 0.3460 - accuracy: 0.8725 - val_loss: 0.3746 - val_accuracy: 0.8630\n",
            "Epoch 4/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.3195 - accuracy: 0.8805 - val_loss: 0.3717 - val_accuracy: 0.8660\n",
            "Epoch 5/10\n",
            "1688/1688 [==============================] - 7s 4ms/step - loss: 0.3035 - accuracy: 0.8886 - val_loss: 0.3682 - val_accuracy: 0.8600\n",
            "Epoch 6/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2881 - accuracy: 0.8935 - val_loss: 0.3879 - val_accuracy: 0.8580\n",
            "Epoch 7/10\n",
            "1688/1688 [==============================] - 6s 4ms/step - loss: 0.2752 - accuracy: 0.8968 - val_loss: 0.3387 - val_accuracy: 0.8750\n",
            "Epoch 8/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2625 - accuracy: 0.9010 - val_loss: 0.3758 - val_accuracy: 0.8760\n",
            "Epoch 9/10\n",
            "1688/1688 [==============================] - 6s 4ms/step - loss: 0.2543 - accuracy: 0.9036 - val_loss: 0.3669 - val_accuracy: 0.8710\n",
            "Epoch 10/10\n",
            "1688/1688 [==============================] - 11s 7ms/step - loss: 0.2445 - accuracy: 0.9074 - val_loss: 0.3280 - val_accuracy: 0.8850\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Evaluate two-layer MLP model\n",
        "two_layer_accuracy, _, _, _ = evaluate_model(two_layer_model, x_test, y_test)\n",
        "\n",
        "# Evaluate three-layer MLP model\n",
        "three_layer_accuracy, _, _, _ = evaluate_model(three_layer_model, x_test, y_test)\n",
        "\n",
        "# Print model comparison\n",
        "print(\"Single-layer MLP Accuracy:\", accuracy)\n",
        "print(\"Two-layer MLP Accuracy:\", two_layer_accuracy)\n",
        "print(\"Three-layer MLP Accuracy:\", three_layer_accuracy)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "aYI-PoIktdLQ",
        "outputId": "137c1fed-0e53-4054-a2b6-d9e50d710bba"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "282/282 [==============================] - 2s 7ms/step\n",
            "282/282 [==============================] - 1s 4ms/step\n",
            "Single-layer MLP Accuracy: 0.8795555555555555\n",
            "Two-layer MLP Accuracy: 0.8806666666666667\n",
            "Three-layer MLP Accuracy: 0.8785555555555555\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Sure! Let's define and train the baseline single-layer MLP model with three different combinations of batch sizes, and then compare their performance.\n",
        "\n",
        "### Define and Train Models with Different Batch Sizes:\n",
        "We'll define and train the baseline single-layer MLP model with three different combinations of batch sizes.\n",
        "\n",
        "#### Combination 1: Batch Size = 32\n",
        "#### Combination 2: Batch Size = 64\n",
        "#### Combination 3: Batch Size = 128\n",
        "\n",
        "Now, let's define and train these models:\n",
        "\n",
        "```python\n",
        "# Define and train model with batch size = 32\n",
        "model_batch_32 = create_single_layer_mlp(input_shape, num_classes)\n",
        "history_batch_32 = train_model(model_batch_32, x_train, y_train, x_val, y_val, batch_size=32)\n",
        "\n",
        "# Define and train model with batch size = 64\n",
        "model_batch_64 = create_single_layer_mlp(input_shape, num_classes)\n",
        "history_batch_64 = train_model(model_batch_64, x_train, y_train, x_val, y_val, batch_size=64)\n",
        "\n",
        "# Define and train model with batch size = 128\n",
        "model_batch_128 = create_single_layer_mlp(input_shape, num_classes)\n",
        "history_batch_128 = train_model(model_batch_128, x_train, y_train, x_val, y_val, batch_size=128)\n",
        "```\n",
        "\n",
        "### Model Comparison:\n",
        "Once the models are trained, we can compare their performance based on evaluation metrics such as accuracy.\n",
        "\n",
        "```python\n",
        "# Evaluate models with different batch sizes\n",
        "accuracy_batch_32, _, _, _ = evaluate_model(model_batch_32, x_test, y_test)\n",
        "accuracy_batch_64, _, _, _ = evaluate_model(model_batch_64, x_test, y_test)\n",
        "accuracy_batch_128, _, _, _ = evaluate_model(model_batch_128, x_test, y_test)\n",
        "\n",
        "# Print model comparison\n",
        "print(\"Single-layer MLP Accuracy (Batch Size = 32):\", accuracy_batch_32)\n",
        "print(\"Single-layer MLP Accuracy (Batch Size = 64):\", accuracy_batch_64)\n",
        "print(\"Single-layer MLP Accuracy (Batch Size = 128):\", accuracy_batch_128)\n",
        "```\n",
        "\n",
        "Explanation:\n",
        "- We define and train three baseline single-layer MLP models with different combinations of batch sizes (32, 64, and 128).\n",
        "- After training, we evaluate the performance of each model using the testing data and compare their accuracies.\n",
        "- The comparison helps us understand how different batch sizes affect the model's performance."
      ],
      "metadata": {
        "id": "s5eo3D-8wVsu"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Define and train model with batch size = 32\n",
        "model_batch_32 = create_single_layer_mlp(input_shape, num_classes)\n",
        "history_batch_32 = train_model(model_batch_32, x_train, y_train, x_val, y_val, batch_size=32)\n",
        "\n",
        "# Define and train model with batch size = 64\n",
        "model_batch_64 = create_single_layer_mlp(input_shape, num_classes)\n",
        "history_batch_64 = train_model(model_batch_64, x_train, y_train, x_val, y_val, batch_size=64)\n",
        "\n",
        "# Define and train model with batch size = 128\n",
        "model_batch_128 = create_single_layer_mlp(input_shape, num_classes)\n",
        "history_batch_128 = train_model(model_batch_128, x_train, y_train, x_val, y_val, batch_size=128)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "EN_emF0iwWw-",
        "outputId": "fdc9a1ae-27ac-401d-cb76-6e96698e6d27"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "1688/1688 [==============================] - 10s 6ms/step - loss: 0.5048 - accuracy: 0.8236 - val_loss: 0.4097 - val_accuracy: 0.8540\n",
            "Epoch 2/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.3824 - accuracy: 0.8630 - val_loss: 0.3971 - val_accuracy: 0.8450\n",
            "Epoch 3/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.3354 - accuracy: 0.8785 - val_loss: 0.3977 - val_accuracy: 0.8570\n",
            "Epoch 4/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.3122 - accuracy: 0.8851 - val_loss: 0.3471 - val_accuracy: 0.8650\n",
            "Epoch 5/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2960 - accuracy: 0.8917 - val_loss: 0.3493 - val_accuracy: 0.8720\n",
            "Epoch 6/10\n",
            "1688/1688 [==============================] - 6s 3ms/step - loss: 0.2784 - accuracy: 0.8983 - val_loss: 0.3583 - val_accuracy: 0.8620\n",
            "Epoch 7/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2705 - accuracy: 0.8989 - val_loss: 0.3780 - val_accuracy: 0.8700\n",
            "Epoch 8/10\n",
            "1688/1688 [==============================] - 6s 4ms/step - loss: 0.2561 - accuracy: 0.9050 - val_loss: 0.3180 - val_accuracy: 0.8780\n",
            "Epoch 9/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2478 - accuracy: 0.9077 - val_loss: 0.3491 - val_accuracy: 0.8630\n",
            "Epoch 10/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2391 - accuracy: 0.9099 - val_loss: 0.3335 - val_accuracy: 0.8710\n",
            "Epoch 1/10\n",
            "844/844 [==============================] - 4s 4ms/step - loss: 0.5267 - accuracy: 0.8177 - val_loss: 0.4307 - val_accuracy: 0.8500\n",
            "Epoch 2/10\n",
            "844/844 [==============================] - 3s 3ms/step - loss: 0.3957 - accuracy: 0.8593 - val_loss: 0.4024 - val_accuracy: 0.8450\n",
            "Epoch 3/10\n",
            "844/844 [==============================] - 3s 3ms/step - loss: 0.3585 - accuracy: 0.8711 - val_loss: 0.3722 - val_accuracy: 0.8670\n",
            "Epoch 4/10\n",
            "844/844 [==============================] - 2s 3ms/step - loss: 0.3275 - accuracy: 0.8810 - val_loss: 0.3690 - val_accuracy: 0.8610\n",
            "Epoch 5/10\n",
            "844/844 [==============================] - 3s 4ms/step - loss: 0.3076 - accuracy: 0.8871 - val_loss: 0.3553 - val_accuracy: 0.8650\n",
            "Epoch 6/10\n",
            "844/844 [==============================] - 3s 3ms/step - loss: 0.2919 - accuracy: 0.8923 - val_loss: 0.3498 - val_accuracy: 0.8560\n",
            "Epoch 7/10\n",
            "844/844 [==============================] - 2s 3ms/step - loss: 0.2807 - accuracy: 0.8971 - val_loss: 0.3336 - val_accuracy: 0.8670\n",
            "Epoch 8/10\n",
            "844/844 [==============================] - 2s 3ms/step - loss: 0.2684 - accuracy: 0.9014 - val_loss: 0.3334 - val_accuracy: 0.8750\n",
            "Epoch 9/10\n",
            "844/844 [==============================] - 2s 3ms/step - loss: 0.2566 - accuracy: 0.9055 - val_loss: 0.3244 - val_accuracy: 0.8760\n",
            "Epoch 10/10\n",
            "844/844 [==============================] - 4s 4ms/step - loss: 0.2495 - accuracy: 0.9082 - val_loss: 0.3845 - val_accuracy: 0.8640\n",
            "Epoch 1/10\n",
            "422/422 [==============================] - 2s 4ms/step - loss: 0.5613 - accuracy: 0.8080 - val_loss: 0.5589 - val_accuracy: 0.7950\n",
            "Epoch 2/10\n",
            "422/422 [==============================] - 2s 4ms/step - loss: 0.4101 - accuracy: 0.8565 - val_loss: 0.4122 - val_accuracy: 0.8550\n",
            "Epoch 3/10\n",
            "422/422 [==============================] - 1s 3ms/step - loss: 0.3696 - accuracy: 0.8691 - val_loss: 0.3819 - val_accuracy: 0.8590\n",
            "Epoch 4/10\n",
            "422/422 [==============================] - 1s 4ms/step - loss: 0.3409 - accuracy: 0.8783 - val_loss: 0.3517 - val_accuracy: 0.8710\n",
            "Epoch 5/10\n",
            "422/422 [==============================] - 1s 3ms/step - loss: 0.3221 - accuracy: 0.8842 - val_loss: 0.3413 - val_accuracy: 0.8650\n",
            "Epoch 6/10\n",
            "422/422 [==============================] - 3s 7ms/step - loss: 0.3045 - accuracy: 0.8895 - val_loss: 0.3597 - val_accuracy: 0.8620\n",
            "Epoch 7/10\n",
            "422/422 [==============================] - 2s 5ms/step - loss: 0.2925 - accuracy: 0.8935 - val_loss: 0.3224 - val_accuracy: 0.8810\n",
            "Epoch 8/10\n",
            "422/422 [==============================] - 1s 4ms/step - loss: 0.2833 - accuracy: 0.8963 - val_loss: 0.3321 - val_accuracy: 0.8750\n",
            "Epoch 9/10\n",
            "422/422 [==============================] - 1s 3ms/step - loss: 0.2725 - accuracy: 0.8995 - val_loss: 0.3266 - val_accuracy: 0.8750\n",
            "Epoch 10/10\n",
            "422/422 [==============================] - 1s 3ms/step - loss: 0.2668 - accuracy: 0.9021 - val_loss: 0.3166 - val_accuracy: 0.8790\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Evaluate models with different batch sizes\n",
        "accuracy_batch_32, _, _, _ = evaluate_model(model_batch_32, x_test, y_test)\n",
        "accuracy_batch_64, _, _, _ = evaluate_model(model_batch_64, x_test, y_test)\n",
        "accuracy_batch_128, _, _, _ = evaluate_model(model_batch_128, x_test, y_test)\n",
        "\n",
        "# Print model comparison\n",
        "print(\"Single-layer MLP Accuracy (Batch Size = 32):\", accuracy_batch_32)\n",
        "print(\"Single-layer MLP Accuracy (Batch Size = 64):\", accuracy_batch_64)\n",
        "print(\"Single-layer MLP Accuracy (Batch Size = 128):\", accuracy_batch_128)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1H9s6eQYxtTY",
        "outputId": "7ac9e9c2-dc3b-46fc-88b6-925cdcf73ccb"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "282/282 [==============================] - 1s 3ms/step\n",
            "282/282 [==============================] - 1s 4ms/step\n",
            "282/282 [==============================] - 1s 3ms/step\n",
            "Single-layer MLP Accuracy (Batch Size = 32): 0.88\n",
            "Single-layer MLP Accuracy (Batch Size = 64): 0.8723333333333333\n",
            "Single-layer MLP Accuracy (Batch Size = 128): 0.8797777777777778\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Of course! Let's define and train the baseline single-layer MLP model with three different combinations of the number of neurons in the hidden layer, and then compare their performance.\n",
        "\n",
        "### Define and Train Models with Different Number of Neurons:\n",
        "We'll define and train the baseline single-layer MLP model with three different combinations of the number of neurons in the hidden layer.\n",
        "\n",
        "#### Combination 1: Number of Neurons = 64\n",
        "#### Combination 2: Number of Neurons = 128\n",
        "#### Combination 3: Number of Neurons = 256\n",
        "\n",
        "Now, let's define and train these models:\n",
        "\n",
        "```python\n",
        "# Define and train model with 64 neurons in the hidden layer\n",
        "model_neurons_64 = create_single_layer_mlp(input_shape, num_classes)\n",
        "history_neurons_64 = train_model(model_neurons_64, x_train, y_train, x_val, y_val)\n",
        "\n",
        "# Define and train model with 128 neurons in the hidden layer\n",
        "model_neurons_128 = create_single_layer_mlp(input_shape, num_classes)\n",
        "history_neurons_128 = train_model(model_neurons_128, x_train, y_train, x_val, y_val)\n",
        "\n",
        "# Define and train model with 256 neurons in the hidden layer\n",
        "model_neurons_256 = create_single_layer_mlp(input_shape, num_classes)\n",
        "history_neurons_256 = train_model(model_neurons_256, x_train, y_train, x_val, y_val)\n",
        "```\n",
        "\n",
        "### Model Comparison:\n",
        "Once the models are trained, we can compare their performance based on evaluation metrics such as accuracy.\n",
        "\n",
        "```python\n",
        "# Evaluate models with different number of neurons\n",
        "accuracy_neurons_64, _, _, _ = evaluate_model(model_neurons_64, x_test, y_test)\n",
        "accuracy_neurons_128, _, _, _ = evaluate_model(model_neurons_128, x_test, y_test)\n",
        "accuracy_neurons_256, _, _, _ = evaluate_model(model_neurons_256, x_test, y_test)\n",
        "\n",
        "# Print model comparison\n",
        "print(\"Single-layer MLP Accuracy (Neurons = 64):\", accuracy_neurons_64)\n",
        "print(\"Single-layer MLP Accuracy (Neurons = 128):\", accuracy_neurons_128)\n",
        "print(\"Single-layer MLP Accuracy (Neurons = 256):\", accuracy_neurons_256)\n",
        "```\n",
        "\n",
        "Explanation:\n",
        "- We define and train three baseline single-layer MLP models with different combinations of the number of neurons in the hidden layer (64, 128, and 256).\n",
        "- After training, we evaluate the performance of each model using the testing data and compare their accuracies.\n",
        "- The comparison helps us understand how different numbers of neurons in the hidden layer affect the model's performance."
      ],
      "metadata": {
        "id": "F2HeGybY0WU8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Define and train model with 64 neurons in the hidden layer\n",
        "model_neurons_64 = create_single_layer_mlp(input_shape, num_classes)\n",
        "history_neurons_64 = train_model(model_neurons_64, x_train, y_train, x_val, y_val)\n",
        "\n",
        "# Define and train model with 128 neurons in the hidden layer\n",
        "model_neurons_128 = create_single_layer_mlp(input_shape, num_classes)\n",
        "history_neurons_128 = train_model(model_neurons_128, x_train, y_train, x_val, y_val)\n",
        "\n",
        "# Define and train model with 256 neurons in the hidden layer\n",
        "model_neurons_256 = create_single_layer_mlp(input_shape, num_classes)\n",
        "history_neurons_256 = train_model(model_neurons_256, x_train, y_train, x_val, y_val)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "sxUcHAI20XCA",
        "outputId": "ade2af89-bcf3-480c-d3fa-ece32e561c80"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "1688/1688 [==============================] - 7s 3ms/step - loss: 0.5028 - accuracy: 0.8228 - val_loss: 0.4306 - val_accuracy: 0.8310\n",
            "Epoch 2/10\n",
            "1688/1688 [==============================] - 8s 4ms/step - loss: 0.3791 - accuracy: 0.8634 - val_loss: 0.3667 - val_accuracy: 0.8610\n",
            "Epoch 3/10\n",
            "1688/1688 [==============================] - 6s 4ms/step - loss: 0.3420 - accuracy: 0.8745 - val_loss: 0.3363 - val_accuracy: 0.8770\n",
            "Epoch 4/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.3145 - accuracy: 0.8845 - val_loss: 0.3435 - val_accuracy: 0.8710\n",
            "Epoch 5/10\n",
            "1688/1688 [==============================] - 6s 3ms/step - loss: 0.2969 - accuracy: 0.8901 - val_loss: 0.3620 - val_accuracy: 0.8580\n",
            "Epoch 6/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2819 - accuracy: 0.8950 - val_loss: 0.3300 - val_accuracy: 0.8800\n",
            "Epoch 7/10\n",
            "1688/1688 [==============================] - 6s 3ms/step - loss: 0.2725 - accuracy: 0.9000 - val_loss: 0.3063 - val_accuracy: 0.8850\n",
            "Epoch 8/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2567 - accuracy: 0.9048 - val_loss: 0.3537 - val_accuracy: 0.8670\n",
            "Epoch 9/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2485 - accuracy: 0.9070 - val_loss: 0.3154 - val_accuracy: 0.8840\n",
            "Epoch 10/10\n",
            "1688/1688 [==============================] - 6s 3ms/step - loss: 0.2380 - accuracy: 0.9115 - val_loss: 0.3316 - val_accuracy: 0.8750\n",
            "Epoch 1/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.5065 - accuracy: 0.8234 - val_loss: 0.4226 - val_accuracy: 0.8430\n",
            "Epoch 2/10\n",
            "1688/1688 [==============================] - 6s 3ms/step - loss: 0.3812 - accuracy: 0.8642 - val_loss: 0.3824 - val_accuracy: 0.8500\n",
            "Epoch 3/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.3403 - accuracy: 0.8758 - val_loss: 0.3748 - val_accuracy: 0.8610\n",
            "Epoch 4/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.3174 - accuracy: 0.8831 - val_loss: 0.3682 - val_accuracy: 0.8650\n",
            "Epoch 5/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2975 - accuracy: 0.8909 - val_loss: 0.3440 - val_accuracy: 0.8740\n",
            "Epoch 6/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2823 - accuracy: 0.8967 - val_loss: 0.3259 - val_accuracy: 0.8790\n",
            "Epoch 7/10\n",
            "1688/1688 [==============================] - 6s 3ms/step - loss: 0.2713 - accuracy: 0.8986 - val_loss: 0.3375 - val_accuracy: 0.8770\n",
            "Epoch 8/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2595 - accuracy: 0.9033 - val_loss: 0.3857 - val_accuracy: 0.8620\n",
            "Epoch 9/10\n",
            "1688/1688 [==============================] - 6s 4ms/step - loss: 0.2473 - accuracy: 0.9075 - val_loss: 0.3499 - val_accuracy: 0.8710\n",
            "Epoch 10/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2388 - accuracy: 0.9094 - val_loss: 0.3596 - val_accuracy: 0.8640\n",
            "Epoch 1/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.5087 - accuracy: 0.8209 - val_loss: 0.4499 - val_accuracy: 0.8300\n",
            "Epoch 2/10\n",
            "1688/1688 [==============================] - 6s 3ms/step - loss: 0.3840 - accuracy: 0.8630 - val_loss: 0.3669 - val_accuracy: 0.8590\n",
            "Epoch 3/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.3437 - accuracy: 0.8751 - val_loss: 0.3631 - val_accuracy: 0.8550\n",
            "Epoch 4/10\n",
            "1688/1688 [==============================] - 6s 3ms/step - loss: 0.3171 - accuracy: 0.8832 - val_loss: 0.3745 - val_accuracy: 0.8600\n",
            "Epoch 5/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2999 - accuracy: 0.8897 - val_loss: 0.3577 - val_accuracy: 0.8620\n",
            "Epoch 6/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2844 - accuracy: 0.8940 - val_loss: 0.3278 - val_accuracy: 0.8740\n",
            "Epoch 7/10\n",
            "1688/1688 [==============================] - 6s 3ms/step - loss: 0.2723 - accuracy: 0.8996 - val_loss: 0.3569 - val_accuracy: 0.8720\n",
            "Epoch 8/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2613 - accuracy: 0.9032 - val_loss: 0.3636 - val_accuracy: 0.8610\n",
            "Epoch 9/10\n",
            "1688/1688 [==============================] - 6s 3ms/step - loss: 0.2516 - accuracy: 0.9065 - val_loss: 0.3431 - val_accuracy: 0.8780\n",
            "Epoch 10/10\n",
            "1688/1688 [==============================] - 6s 3ms/step - loss: 0.2426 - accuracy: 0.9097 - val_loss: 0.3367 - val_accuracy: 0.8790\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Evaluate models with different number of neurons\n",
        "accuracy_neurons_64, _, _, _ = evaluate_model(model_neurons_64, x_test, y_test)\n",
        "accuracy_neurons_128, _, _, _ = evaluate_model(model_neurons_128, x_test, y_test)\n",
        "accuracy_neurons_256, _, _, _ = evaluate_model(model_neurons_256, x_test, y_test)\n",
        "\n",
        "# Print model comparison\n",
        "print(\"Single-layer MLP Accuracy (Neurons = 64):\", accuracy_neurons_64)\n",
        "print(\"Single-layer MLP Accuracy (Neurons = 128):\", accuracy_neurons_128)\n",
        "print(\"Single-layer MLP Accuracy (Neurons = 256):\", accuracy_neurons_256)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "O5GnXDmC0b32",
        "outputId": "f06e626b-b63f-4906-8c00-60ecddd286d5"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "282/282 [==============================] - 0s 2ms/step\n",
            "282/282 [==============================] - 0s 2ms/step\n",
            "282/282 [==============================] - 0s 1ms/step\n",
            "Single-layer MLP Accuracy (Neurons = 64): 0.8773333333333333\n",
            "Single-layer MLP Accuracy (Neurons = 128): 0.8754444444444445\n",
            "Single-layer MLP Accuracy (Neurons = 256): 0.8813333333333333\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Certainly! Let's define and train the baseline single-layer MLP model with three different optimizers (SGD, Adam, and RMSprop), and then compare their performance.\n",
        "\n",
        "### Define and Train Models with Different Optimizers:\n",
        "We'll define and train the baseline single-layer MLP model with three different optimizers: SGD, Adam, and RMSprop.\n",
        "\n",
        "Now, let's define and train these models:\n",
        "\n",
        "```python\n",
        "from tensorflow.keras.optimizers import SGD, Adam, RMSprop\n",
        "\n",
        "# Define and train model with SGD optimizer\n",
        "def train_model_sgd(model, x_train, y_train, x_val, y_val, batch_size=32, epochs=10):\n",
        "    # One-hot encode the class labels\n",
        "    y_train_encoded = to_categorical(y_train)\n",
        "    y_val_encoded = to_categorical(y_val)\n",
        "    # Compile the model with SGD optimizer\n",
        "    model.compile(loss='categorical_crossentropy', optimizer=SGD(), metrics=['accuracy'])\n",
        "    # Train the model on the training data\n",
        "    history = model.fit(x_train, y_train_encoded, batch_size=batch_size, epochs=epochs, validation_data=(x_val, y_val_encoded))\n",
        "    return history\n",
        "\n",
        "# Define and train model with Adam optimizer\n",
        "def train_model_adam(model, x_train, y_train, x_val, y_val, batch_size=32, epochs=10):\n",
        "    # One-hot encode the class labels\n",
        "    y_train_encoded = to_categorical(y_train)\n",
        "    y_val_encoded = to_categorical(y_val)\n",
        "    # Compile the model with Adam optimizer\n",
        "    model.compile(loss='categorical_crossentropy', optimizer=Adam(), metrics=['accuracy'])\n",
        "    # Train the model on the training data\n",
        "    history = model.fit(x_train, y_train_encoded, batch_size=batch_size, epochs=epochs, validation_data=(x_val, y_val_encoded))\n",
        "    return history\n",
        "\n",
        "# Define and train model with RMSprop optimizer\n",
        "def train_model_rmsprop(model, x_train, y_train, x_val, y_val, batch_size=32, epochs=10):\n",
        "    # One-hot encode the class labels\n",
        "    y_train_encoded = to_categorical(y_train)\n",
        "    y_val_encoded = to_categorical(y_val)\n",
        "    # Compile the model with RMSprop optimizer\n",
        "    model.compile(loss='categorical_crossentropy', optimizer=RMSprop(), metrics=['accuracy'])\n",
        "    # Train the model on the training data\n",
        "    history = model.fit(x_train, y_train_encoded, batch_size=batch_size, epochs=epochs, validation_data=(x_val, y_val_encoded))\n",
        "    return history\n",
        "\n",
        "# Train model with SGD optimizer\n",
        "model_sgd = create_single_layer_mlp(input_shape, num_classes)\n",
        "history_sgd = train_model_sgd(model_sgd, x_train, y_train, x_val, y_val)\n",
        "\n",
        "# Train model with Adam optimizer\n",
        "model_adam = create_single_layer_mlp(input_shape, num_classes)\n",
        "history_adam = train_model_adam(model_adam, x_train, y_train, x_val, y_val)\n",
        "\n",
        "# Train model with RMSprop optimizer\n",
        "model_rmsprop = create_single_layer_mlp(input_shape, num_classes)\n",
        "history_rmsprop = train_model_rmsprop(model_rmsprop, x_train, y_train, x_val, y_val)\n",
        "```\n",
        "\n",
        "### Model Comparison:\n",
        "Once the models are trained, we can compare their performance based on evaluation metrics such as accuracy.\n",
        "\n",
        "```python\n",
        "# Evaluate models with different optimizers\n",
        "accuracy_sgd, _, _, _ = evaluate_model(model_sgd, x_test, y_test)\n",
        "accuracy_adam, _, _, _ = evaluate_model(model_adam, x_test, y_test)\n",
        "accuracy_rmsprop, _, _, _ = evaluate_model(model_rmsprop, x_test, y_test)\n",
        "\n",
        "# Print model comparison\n",
        "print(\"Single-layer MLP Accuracy (SGD):\", accuracy_sgd)\n",
        "print(\"Single-layer MLP Accuracy (Adam):\", accuracy_adam)\n",
        "print(\"Single-layer MLP Accuracy (RMSprop):\", accuracy_rmsprop)\n",
        "```\n",
        "\n",
        "Explanation:\n",
        "- We define and train three baseline single-layer MLP models with different optimizers (SGD, Adam, and RMSprop).\n",
        "- After training, we evaluate the performance of each model using the testing data and compare their accuracies.\n",
        "- The comparison helps us understand how different optimizers affect the model's performance."
      ],
      "metadata": {
        "id": "tciYiFB-2Qtb"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from tensorflow.keras.optimizers import SGD, Adam, RMSprop\n",
        "\n",
        "# Define and train model with SGD optimizer\n",
        "def train_model_sgd(model, x_train, y_train, x_val, y_val, batch_size=32, epochs=10):\n",
        "    # One-hot encode the class labels\n",
        "    y_train_encoded = to_categorical(y_train)\n",
        "    y_val_encoded = to_categorical(y_val)\n",
        "    # Compile the model with SGD optimizer\n",
        "    model.compile(loss='categorical_crossentropy', optimizer=SGD(), metrics=['accuracy'])\n",
        "    # Train the model on the training data\n",
        "    history = model.fit(x_train, y_train_encoded, batch_size=batch_size, epochs=epochs, validation_data=(x_val, y_val_encoded))\n",
        "    return history\n",
        "\n",
        "# Define and train model with Adam optimizer\n",
        "def train_model_adam(model, x_train, y_train, x_val, y_val, batch_size=32, epochs=10):\n",
        "    # One-hot encode the class labels\n",
        "    y_train_encoded = to_categorical(y_train)\n",
        "    y_val_encoded = to_categorical(y_val)\n",
        "    # Compile the model with Adam optimizer\n",
        "    model.compile(loss='categorical_crossentropy', optimizer=Adam(), metrics=['accuracy'])\n",
        "    # Train the model on the training data\n",
        "    history = model.fit(x_train, y_train_encoded, batch_size=batch_size, epochs=epochs, validation_data=(x_val, y_val_encoded))\n",
        "    return history\n",
        "\n",
        "# Define and train model with RMSprop optimizer\n",
        "def train_model_rmsprop(model, x_train, y_train, x_val, y_val, batch_size=32, epochs=10):\n",
        "    # One-hot encode the class labels\n",
        "    y_train_encoded = to_categorical(y_train)\n",
        "    y_val_encoded = to_categorical(y_val)\n",
        "    # Compile the model with RMSprop optimizer\n",
        "    model.compile(loss='categorical_crossentropy', optimizer=RMSprop(), metrics=['accuracy'])\n",
        "    # Train the model on the training data\n",
        "    history = model.fit(x_train, y_train_encoded, batch_size=batch_size, epochs=epochs, validation_data=(x_val, y_val_encoded))\n",
        "    return history\n",
        "\n",
        "# Train model with SGD optimizer\n",
        "model_sgd = create_single_layer_mlp(input_shape, num_classes)\n",
        "history_sgd = train_model_sgd(model_sgd, x_train, y_train, x_val, y_val)\n",
        "\n",
        "# Train model with Adam optimizer\n",
        "model_adam = create_single_layer_mlp(input_shape, num_classes)\n",
        "history_adam = train_model_adam(model_adam, x_train, y_train, x_val, y_val)\n",
        "\n",
        "# Train model with RMSprop optimizer\n",
        "model_rmsprop = create_single_layer_mlp(input_shape, num_classes)\n",
        "history_rmsprop = train_model_rmsprop(model_rmsprop, x_train, y_train, x_val, y_val)\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zDk1Y6g82QaQ",
        "outputId": "69721961-c711-4466-d801-e3fa99a01182"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "1688/1688 [==============================] - 5s 2ms/step - loss: 0.7596 - accuracy: 0.7583 - val_loss: 0.5986 - val_accuracy: 0.7970\n",
            "Epoch 2/10\n",
            "1688/1688 [==============================] - 4s 2ms/step - loss: 0.5251 - accuracy: 0.8219 - val_loss: 0.5112 - val_accuracy: 0.8260\n",
            "Epoch 3/10\n",
            "1688/1688 [==============================] - 4s 2ms/step - loss: 0.4790 - accuracy: 0.8358 - val_loss: 0.4770 - val_accuracy: 0.8240\n",
            "Epoch 4/10\n",
            "1688/1688 [==============================] - 4s 2ms/step - loss: 0.4536 - accuracy: 0.8442 - val_loss: 0.4756 - val_accuracy: 0.8200\n",
            "Epoch 5/10\n",
            "1688/1688 [==============================] - 4s 2ms/step - loss: 0.4358 - accuracy: 0.8480 - val_loss: 0.4452 - val_accuracy: 0.8340\n",
            "Epoch 6/10\n",
            "1688/1688 [==============================] - 4s 2ms/step - loss: 0.4205 - accuracy: 0.8553 - val_loss: 0.4258 - val_accuracy: 0.8470\n",
            "Epoch 7/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.4090 - accuracy: 0.8573 - val_loss: 0.4225 - val_accuracy: 0.8420\n",
            "Epoch 8/10\n",
            "1688/1688 [==============================] - 4s 2ms/step - loss: 0.3992 - accuracy: 0.8611 - val_loss: 0.4105 - val_accuracy: 0.8450\n",
            "Epoch 9/10\n",
            "1688/1688 [==============================] - 4s 2ms/step - loss: 0.3897 - accuracy: 0.8645 - val_loss: 0.4086 - val_accuracy: 0.8560\n",
            "Epoch 10/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.3807 - accuracy: 0.8668 - val_loss: 0.4129 - val_accuracy: 0.8520\n",
            "Epoch 1/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.4992 - accuracy: 0.8238 - val_loss: 0.4165 - val_accuracy: 0.8450\n",
            "Epoch 2/10\n",
            "1688/1688 [==============================] - 6s 3ms/step - loss: 0.3772 - accuracy: 0.8651 - val_loss: 0.4121 - val_accuracy: 0.8490\n",
            "Epoch 3/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.3371 - accuracy: 0.8775 - val_loss: 0.3730 - val_accuracy: 0.8500\n",
            "Epoch 4/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.3124 - accuracy: 0.8859 - val_loss: 0.3396 - val_accuracy: 0.8660\n",
            "Epoch 5/10\n",
            "1688/1688 [==============================] - 6s 3ms/step - loss: 0.2926 - accuracy: 0.8921 - val_loss: 0.3300 - val_accuracy: 0.8780\n",
            "Epoch 6/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2802 - accuracy: 0.8961 - val_loss: 0.3396 - val_accuracy: 0.8750\n",
            "Epoch 7/10\n",
            "1688/1688 [==============================] - 7s 4ms/step - loss: 0.2641 - accuracy: 0.9016 - val_loss: 0.3245 - val_accuracy: 0.8810\n",
            "Epoch 8/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2531 - accuracy: 0.9057 - val_loss: 0.3540 - val_accuracy: 0.8670\n",
            "Epoch 9/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2446 - accuracy: 0.9093 - val_loss: 0.3253 - val_accuracy: 0.8770\n",
            "Epoch 10/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2360 - accuracy: 0.9118 - val_loss: 0.3144 - val_accuracy: 0.8820\n",
            "Epoch 1/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.5208 - accuracy: 0.8142 - val_loss: 0.4141 - val_accuracy: 0.8450\n",
            "Epoch 2/10\n",
            "1688/1688 [==============================] - 4s 3ms/step - loss: 0.3843 - accuracy: 0.8624 - val_loss: 0.4384 - val_accuracy: 0.8330\n",
            "Epoch 3/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.3482 - accuracy: 0.8734 - val_loss: 0.3396 - val_accuracy: 0.8690\n",
            "Epoch 4/10\n",
            "1688/1688 [==============================] - 4s 3ms/step - loss: 0.3300 - accuracy: 0.8821 - val_loss: 0.3987 - val_accuracy: 0.8530\n",
            "Epoch 5/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.3132 - accuracy: 0.8889 - val_loss: 0.3881 - val_accuracy: 0.8670\n",
            "Epoch 6/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.3020 - accuracy: 0.8924 - val_loss: 0.3599 - val_accuracy: 0.8730\n",
            "Epoch 7/10\n",
            "1688/1688 [==============================] - 4s 3ms/step - loss: 0.2931 - accuracy: 0.8963 - val_loss: 0.3958 - val_accuracy: 0.8630\n",
            "Epoch 8/10\n",
            "1688/1688 [==============================] - 5s 3ms/step - loss: 0.2829 - accuracy: 0.8991 - val_loss: 0.4247 - val_accuracy: 0.8650\n",
            "Epoch 9/10\n",
            "1688/1688 [==============================] - 4s 3ms/step - loss: 0.2756 - accuracy: 0.9037 - val_loss: 0.3528 - val_accuracy: 0.8750\n",
            "Epoch 10/10\n",
            "1688/1688 [==============================] - 4s 3ms/step - loss: 0.2710 - accuracy: 0.9042 - val_loss: 0.4044 - val_accuracy: 0.8710\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Evaluate models with different optimizers\n",
        "accuracy_sgd, _, _, _ = evaluate_model(model_sgd, x_test, y_test)\n",
        "accuracy_adam, _, _, _ = evaluate_model(model_adam, x_test, y_test)\n",
        "accuracy_rmsprop, _, _, _ = evaluate_model(model_rmsprop, x_test, y_test)\n",
        "\n",
        "# Print model comparison\n",
        "print(\"Single-layer MLP Accuracy (SGD):\", accuracy_sgd)\n",
        "print(\"Single-layer MLP Accuracy (Adam):\", accuracy_adam)\n",
        "print(\"Single-layer MLP Accuracy (RMSprop):\", accuracy_rmsprop)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6ZTeEUId4M6O",
        "outputId": "931efa1b-7332-4094-a88d-4ebfbc920733"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "282/282 [==============================] - 1s 2ms/step\n",
            "282/282 [==============================] - 1s 3ms/step\n",
            "282/282 [==============================] - 0s 1ms/step\n",
            "Single-layer MLP Accuracy (SGD): 0.8492222222222222\n",
            "Single-layer MLP Accuracy (Adam): 0.8816666666666667\n",
            "Single-layer MLP Accuracy (RMSprop): 0.8663333333333333\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import Dense, Dropout\n",
        "from tensorflow.keras.optimizers import Adam\n",
        "from tensorflow.keras.callbacks import EarlyStopping\n",
        "from tensorflow.keras.datasets import fashion_mnist\n",
        "from tensorflow.keras.utils import to_categorical\n",
        "\n",
        "# Load Fashion MNIST dataset\n",
        "(x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()\n",
        "\n",
        "# Pre-process the data\n",
        "x_train = x_train.reshape((-1, 28 * 28)) / 255.0\n",
        "x_test = x_test.reshape((-1, 28 * 28)) / 255.0\n",
        "y_train = to_categorical(y_train)\n",
        "y_test = to_categorical(y_test)\n",
        "\n",
        "# Define the model\n",
        "model = Sequential()\n",
        "model.add(Dense(512, activation='relu', input_shape=(28 * 28,)))\n",
        "model.add(Dropout(0.5))  # Dropout layer to prevent overfitting\n",
        "model.add(Dense(256, activation='relu'))\n",
        "model.add(Dropout(0.5))\n",
        "model.add(Dense(128, activation='relu'))\n",
        "model.add(Dropout(0.5))\n",
        "model.add(Dense(10, activation='softmax'))\n",
        "\n",
        "# Compile the model with Adam optimizer and set learning rate\n",
        "optimizer = Adam(lr=0.001)\n",
        "model.compile(optimizer=optimizer, loss='categorical_crossentropy', metrics=['accuracy'])\n",
        "\n",
        "# Define early stopping callback to prevent overfitting\n",
        "early_stopping = EarlyStopping(monitor='val_loss', patience=5, restore_best_weights=True)\n",
        "\n",
        "# Train the model with early stopping\n",
        "history = model.fit(x_train, y_train, epochs=50, batch_size=128, validation_split=0.2, callbacks=[early_stopping])\n",
        "\n",
        "# Evaluate the model\n",
        "accuracy = model.evaluate(x_test, y_test, verbose=0)[1]\n",
        "print(\"Test Accuracy:\", accuracy)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RBWWeonH89iZ",
        "outputId": "4d772baf-fecc-4d55-f40e-9a61dcf6cf7c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "WARNING:absl:`lr` is deprecated in Keras optimizer, please use `learning_rate` or use the legacy optimizer, e.g.,tf.keras.optimizers.legacy.Adam.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/50\n",
            "375/375 [==============================] - 9s 19ms/step - loss: 0.8545 - accuracy: 0.6902 - val_loss: 0.4771 - val_accuracy: 0.8293\n",
            "Epoch 2/50\n",
            "375/375 [==============================] - 5s 15ms/step - loss: 0.5447 - accuracy: 0.8079 - val_loss: 0.4251 - val_accuracy: 0.8425\n",
            "Epoch 3/50\n",
            "375/375 [==============================] - 6s 16ms/step - loss: 0.4947 - accuracy: 0.8276 - val_loss: 0.3940 - val_accuracy: 0.8543\n",
            "Epoch 4/50\n",
            "375/375 [==============================] - 6s 15ms/step - loss: 0.4701 - accuracy: 0.8349 - val_loss: 0.3898 - val_accuracy: 0.8608\n",
            "Epoch 5/50\n",
            "375/375 [==============================] - 5s 15ms/step - loss: 0.4457 - accuracy: 0.8424 - val_loss: 0.3727 - val_accuracy: 0.8643\n",
            "Epoch 6/50\n",
            "375/375 [==============================] - 6s 16ms/step - loss: 0.4310 - accuracy: 0.8485 - val_loss: 0.3694 - val_accuracy: 0.8656\n",
            "Epoch 7/50\n",
            "375/375 [==============================] - 5s 14ms/step - loss: 0.4210 - accuracy: 0.8503 - val_loss: 0.3595 - val_accuracy: 0.8672\n",
            "Epoch 8/50\n",
            "375/375 [==============================] - 6s 17ms/step - loss: 0.4122 - accuracy: 0.8549 - val_loss: 0.3510 - val_accuracy: 0.8690\n",
            "Epoch 9/50\n",
            "375/375 [==============================] - 5s 14ms/step - loss: 0.3986 - accuracy: 0.8590 - val_loss: 0.3455 - val_accuracy: 0.8742\n",
            "Epoch 10/50\n",
            "375/375 [==============================] - 6s 17ms/step - loss: 0.3876 - accuracy: 0.8608 - val_loss: 0.3420 - val_accuracy: 0.8760\n",
            "Epoch 11/50\n",
            "375/375 [==============================] - 5s 14ms/step - loss: 0.3864 - accuracy: 0.8611 - val_loss: 0.3381 - val_accuracy: 0.8763\n",
            "Epoch 12/50\n",
            "375/375 [==============================] - 6s 17ms/step - loss: 0.3802 - accuracy: 0.8658 - val_loss: 0.3360 - val_accuracy: 0.8772\n",
            "Epoch 13/50\n",
            "375/375 [==============================] - 5s 14ms/step - loss: 0.3729 - accuracy: 0.8667 - val_loss: 0.3362 - val_accuracy: 0.8783\n",
            "Epoch 14/50\n",
            "375/375 [==============================] - 6s 16ms/step - loss: 0.3660 - accuracy: 0.8694 - val_loss: 0.3343 - val_accuracy: 0.8792\n",
            "Epoch 15/50\n",
            "375/375 [==============================] - 6s 15ms/step - loss: 0.3622 - accuracy: 0.8697 - val_loss: 0.3308 - val_accuracy: 0.8808\n",
            "Epoch 16/50\n",
            "375/375 [==============================] - 5s 15ms/step - loss: 0.3593 - accuracy: 0.8694 - val_loss: 0.3299 - val_accuracy: 0.8802\n",
            "Epoch 17/50\n",
            "375/375 [==============================] - 6s 17ms/step - loss: 0.3549 - accuracy: 0.8726 - val_loss: 0.3447 - val_accuracy: 0.8759\n",
            "Epoch 18/50\n",
            "375/375 [==============================] - 5s 14ms/step - loss: 0.3522 - accuracy: 0.8719 - val_loss: 0.3283 - val_accuracy: 0.8863\n",
            "Epoch 19/50\n",
            "375/375 [==============================] - 6s 17ms/step - loss: 0.3476 - accuracy: 0.8754 - val_loss: 0.3216 - val_accuracy: 0.8844\n",
            "Epoch 20/50\n",
            "375/375 [==============================] - 5s 14ms/step - loss: 0.3462 - accuracy: 0.8748 - val_loss: 0.3288 - val_accuracy: 0.8785\n",
            "Epoch 21/50\n",
            "375/375 [==============================] - 6s 17ms/step - loss: 0.3430 - accuracy: 0.8777 - val_loss: 0.3228 - val_accuracy: 0.8852\n",
            "Epoch 22/50\n",
            "375/375 [==============================] - 6s 15ms/step - loss: 0.3347 - accuracy: 0.8796 - val_loss: 0.3266 - val_accuracy: 0.8808\n",
            "Epoch 23/50\n",
            "375/375 [==============================] - 6s 16ms/step - loss: 0.3367 - accuracy: 0.8799 - val_loss: 0.3230 - val_accuracy: 0.8858\n",
            "Epoch 24/50\n",
            "375/375 [==============================] - 5s 14ms/step - loss: 0.3305 - accuracy: 0.8798 - val_loss: 0.3251 - val_accuracy: 0.8829\n",
            "Test Accuracy: 0.878000020980835\n"
          ]
        }
      ]
    }
  ]
}